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ABSTRACT 

This guide is intended for local- and state-level 
vocational education administrators, planners, and evaluators who are 
responsible for making program planning and evaluation findings and 
decisions geared toward increasing program relevance. The method 
differs from more traditional approaches in its heavy reliance upon 
the selection and application of quantitative as opposed to 
qualitative, judgmental, or intuitive data. Individuals using this 
evaluation method must (1) explicitly specify the relative importance 
they attribute to planning and evaluation components and to the kinds 
of data they select for use with the methodology and (2) value data 
quality. The introductory section of the guide includes an overview 
of the method, background information on the development of the 
method, and a description of the guide's intended audience. The 
second chapter covers design specifications, the procedures entailed 
in using the method (the information and data selection framework and 
the scoring and ranking processes used), and the role of the 
microcomputer in increasing the efficiency of the method's ranking 
process. The third section, which describes one example of 
implementation of the method, includes sections dealing with the 
following: site, ranking process, and grand labor market area context 
description (context for employment, employers' needs for workers, 
people|s needs for jobs, and capacity to meet needs for training). 
Appendixes include examples of the following: information categories; 
performance indicators, measures, and scores; and databases and 
sources. (MN) 
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Executive Summary 



This publication describes a program planning and evaluation method for increasing the rele- 
vance of vocational education offerings in meeting employment needs of students, employers, and 
labor market areas. Both quantitative data and explicit value judgements are used for comparing 
and ranking ongoing programs or candidate occupations. 

The method is different from more traditional approaches in several v*-jys. It relies heavily on 
selecting and applying quantitative as opposed to qualitative Judgmental or intuitive data. How- 
ever, the most notable differences are that users (1) must explicitly specify the relative importance 
they attribute to planning and evaluation components (e.g., the employment context, educational 
process, benefits of vocational education) and to the kinds of data they select for use with the 
methodology and (2) must value data quality. 

Personsdoing data-based planning and evaluation may find itverydifficulttocomparedifferent 
kinds of useful data. For example, how can programs be compared when (1) program A has an 85 
percent completion rate and a 63 percent placement rate, and there is an annual average demand 
for 1,132 persons in the occupation for which training is being offered, whereas (2) program Bhas a 93 
percent completion rate and a 49 percent placement rate, and there is an annual average demand 
for 978 persons in the occupation. The question becomes how to use these different kinds of data to 
compare program A to program B todetermine which is more relevant in meeting employment needs. 

The program planning and evaluation method described in this publication provides a relatively 
simple way to compare data by using two data normalizing procedures. Data are given a score of 
either one or zero depending on whether they pass or fail a prespecified criterion; or data are scored on 
an arbitrary range (e.g. one to three points or one to five points) depending on multiple criteria set for 
each datum used in the planning or evaluation process. The method comprises three separate 
components: (1) an information and data selection framework, (2) a scoring process, and (3) a 
ranking process. 

The Information and data selection framework provides a procedure for systematically selecting 
appropriate quantitative data. The framework includes five steps. Users must first describe their 
planning or evaluation problem in some detail. This helps users to select data that are appropriate to 
the planning or evaluation task. Rve components of a vocational education planning and evaluation 
process (employment context, vocational education process, outputs, outcomes, and benefits) are 
used as an organizing scheme for selecting data for use with the method. Next, users identify 
information categories for each component (e.g., supply/demand and employment conditions for 
the employment context component) and performance indicators for each information category 
(e.g., whether or not affirmative action is practiced by local employers in hiring for this occupation 
becomes a performance indicator for the information category of employment conditions). Finally 
users prepare performance measures for each indicator, which are statements that lend themselves to 
quantification. 

The steps required to implement the framework are explained in detail in this publication. Exhibits 
are used to depict the steps. In addition, two appendixes provide examples of information categories, 
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performance indicators, and performance measures as weii as sources for obtaining program plan- 
ning and evaluation data 



The scoring process requires two steps. First, a normalizing procedure is selected for use with the 
method. Then, performance measure statements are formatted to be compatible with the selected 
normalizing procedure. 

The ranking process includes several steps. Users are shown how to (1) assign values (i.e., 
weightings of importance) to components of the framework; (2) adjust normalized data (perfor- 
mance measure) scores for their perceived quality; (3) multiply the weighting assigned to each 
performance measure by its adjusted normalized score (where applicable) and then add the 
recomputed scores for each candidate occupation (or occupational cluster) where the methodol- 
ogy is used for program planning or for ongoing offerings (or clusters) where the methodology is used 
for program evaluation; and (4) rank candidate occupations and ongoing programs based on the 
total recomputed score achieved by each. 

The candidate occupations with the highest recomputed scores would, therefore, best meet 
employment needs and deserve consideration for program development and implementation. The 
ongoing offerings with the lowest recomputed scores would most need program improvement to 
increase relevance in meeting employment needs or be candidates for termination. 

Different individuals or groups (e.g., educators and employers) may hold different values about 
the importance of planning and evaluation components and about the quality of data used. The 
method is designed todeai with this possibility by enabling userstoexamine the effects of differingsets 
of values on rankings. This enables them to test the consequences of different sets of values based on 
their assumptions about present and future employment, education, and economic conditions. 

The ranking process can be used with a spreadsheet program and a microcomputer, which help 
users determinethe effects on rankings if values are altered to satisfy group Aor group Bor if weightings 
are altered to reflect different assumptions about the future. A spreadsheet program can almost 
instantaneously compute new total scores and ranks of candidate occupations or ongoing programs 
as values (i.e., weightings of importance) are changed. Although the computation can be done 
manually, a spreadsheet program makes the ranking process easier. 

There may be difficulties in implementing the method if administrators are not familiar with, or 
comfortable in using, quantitative data for program planning and evaluation. In such situations, it is 
highly likely that this publication aione will not be sufficient to enable them to implement the method, 
instead, they may require some assistance or additional documentation, in other instances, school 
personnel may be ready to implement the method but may not have desirable data available. These 
issues are discussed in the body of the publication. r 

Although the method has not been fieid-tested, an external panel was asked to review it. The 
panel members found that it was conceptually sound and deserved consideration for implementa- 
tion in vocational education institutions and agencies. 
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Chapter i 
introduction 



This publication describes a program planning and evaluation method designed to use quantita- 
tive data and explicit value judgements for accomplishing the following: 

• Comparing and ranking candidate occupations being considered for program develop- 
ment and implementation in order to determine which of them best meet employment 
needs of students, employers, and labor markets areas 

• Comparing and ranking ongoing vocational education programs to determine which of 
them most need improvement or should be considered for termination because they 
least meet employment needs 

The method is intended to complement rather than replace, traditional methods of planning and 
evaluating vocational education. 

The National Center for Research in Vocational Education has completed a number of research 
and development projects that provide a better understanding of the progress and problems, 
nationally, associated with vocational education program planning and evaluation (e.g., Starr et al. 
1 976; Starr et al. 1 979; Starr et al. 1 980; Starr et al. 1 981 ; Starr, Merz, and Zahnlser 1 982). One outcome of 
these National Center studies is the recognition that there continues to be a need by practitioners for 
planning and evaluation methods that make use of the best quantitative data available. 

Vocational educators have not been overly enthusiastic about quantitative planning and eval- 
uation methods. Instead, planning and evaluating methods have traditionally relied on expert 
judgments, observations, and intuition. Where differences of opinion exist about planning and evalua- 
tion findings, it is often uncertain whethersuch findings have resulted from differences among planners 
or evaluators with respect to their expertise or because they were looking at different things or the same 
things differently. 

Federal vocational education legislation since 1963 has promoted the application of the best 
data available for planning and evaluation. In response, many state vocational education agencies 
and local education agencies and Institutions have put in place more sophisticated management 
information systems. Nevertheless, quantitative data are still used more frequently for reporting pur- 
poses than for program planning and evaluation. 

Part of the reason for the lack of data-based program planning and evaluation methods may be 
that many vocational education administrators are unsure about how to select, value, compare, and 
applythedifferentkindsof data available to them. The present method is specifically designed to deal 
with these four issues. 
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Among the characteristics that distinguish the present method from traditional ones are the 
following: 



• It is data-based. 

• A framework helps users select appropriate planning and evaluation data 

• Quantitative data can be compared by "normalizing" them. That is, regardless of their kind or 
actual extent, all data are scored with one or zero or a simple range of scores (e.g., one to 
three or one to five). The normalization process greatly simplifies comparisons of dissimilar 
data. 

• Value judgments about the relative contribution of different kinds of data to the planning or 
evaluation process can be explicitly incorporated. 

• Differences in the perceived quality of the selected data can be explicitly accounted for. 

• Candidate occupations or ongoing vocational education offerings can be ranked, tak,; lg 
into account both quantitative but normalized data and their importance and quality. Thus, 
the present method is both data- and value-driven. 

• Rankings can be established based on alternative sets cf value judgments by different 
individuals or groups. Use of a microcomputer with spreadsheet software will enhance the 
speed of establishing rankings based on alternative sets of values. 

□veruiew of the method 

The program planning and evaluation method consists of three components: 

1. An Information and data selection framework consisting of procedures for systematically 
selecting information and data that users n * •, want to incorporate in the method for doing 
either planning or evaluation. 

2. A scoring process that involves selecting a data-normalizing procedure and formatting 
compatible data descriptions. 

3. A ranking process for determining which candidate occupations are best suited for program 
development and implementation and which ongoing programs should be considered for 
termination or Improvement. 

Figure 1 presents the steps required to Implement the method. These steps will be explained in 
chapter 2. 

Background for the Development of the method 

The method was developed in response to a continuing national intention for improved planning 
and evaluation in vocational education. This intention has been expressed in federal legislation since 
1963 (i.e.. the Vocational Education Act of 1963 and as subsequently amended, the vocational 
education provisions of the Education Amendments of 1976. and the Cari D. Perkins Vocational 
Education Act of 1984). 
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I. THE INFORMATION SELECTION FRAMEWORK 



A DEFINE THE PLANNING OR EVALUATION PROBLEM 

1. STATE THE MAJOR REASONS FOR DOING PLANNING OR EVALUATION 

2. FORMULATE A CONTEXT DESCRIPTION 

a. DESCRIBE ECONOMIC. SOCIAL. EDUCATIONAL, AND DEMOGRAPHIC CONDITIONS WITHIN 
THE LABOR MARKET AREA(S) BEING SERVED 

b. DESCRIBE INSTITUTIONAL RESOURCES. CAPABILITIES. AND CONSTRAINTS 



B. LIST THE FIVE PLANNING AND EVALUATION COMPONENTS (e.g.. EMPLOYMENT CONTEXT. VOCA- 
TIONAL EDUCATION PROCESSES) 



C. SPECIFY INFORMATION CATEGORIES FOR EACH PLANNING AND EVALUATION COMPONENT (e.g.. 
FOR THE EMPLOYMENT CONTEXT COMPONENT-EMPLOYMENT OPPORTUNITIES. EMPLOYMENT 
CONDITIONS) 



D. ASSIGN ONEOR MORE PERFORMANCE INDICATORSTO EACH INFORMATION CATEGORY (e.g.. FOR 
THE INFORMATION CATEGORY OF EMPLOYMENT OPPORTUNITIES— WHETHER CURRENT OR PRO- 
JECTED DEMAND FOR NEW WORKERS IN A CANDIDATE OCCUPATION IS SUFFICIENT FOR IMPLEMENT- 
ING A PROGRAM) 



E. SELECT ONE OR MORE PERFORMANCE MEASURES FOR EACH PERFORMANCE INDICATOR (e.g.. FOR 
THE INDICATOR "WHETHER CURRENT OR PROJECTED DEMAND FOR NEW WORKERSIN A CANDIDATE 
OCCUPATION IS SUFFICIENT FOR IMPLEMENTING A PROGRAM"— IF THIS CANDIDATE OCCUPATION 
IS/IS NOT AMONG THE 10 FASTEST-GROWING OCCUPATIONS IN THIS LABOR MARKET AREA) 



II. THE SCORING PROCESS 



A SELECT A NORMALIZING PROCEDURE FOR COMPARING PERFORMANCE MEASURE DATA 

1. SELECT EITHER THE PASS/FAIL NORMALIZING PROCEDURE OR THE RANGE NORMALIZING 
PROCEDURE FOR SCORING PERFORMANCE MEASURES 



B. PREPARE PERFORMANCE MEASURE STATEMENTS THAT ARE COMPATIBLE WITH THE SELECTED NOR- 
MALIZING PROCEDURE (E.G.. FOR PASS/FAIL NORMALIZATION PROCEDURE-IF THIS CANDIDATE IS 
AMONG THE 1 0 FASTEST-GROWING OCCUPATIONS IN THIS LABOR MARKET AREA SCORE 1; IF NOT. 
SCORE 0. FOR RANGE NORMALIZATION PROCEDURE-THIS CANDIDATE IS AMONG THE 10 FASTEST- 
GROWING OCCUPATIONS. SCORE 3; THIS CANDIDATE IS NOT AMONG THE 10 FASTEST-GROWING 
ONES. BUT IS EXHIBITING GROWTH. SCORE 2; THIS CANDIDATE'S PROJECTED RATE OF GROWTH IS 
EXPECTED TO BE CONSTANT OR ELSE DECLINE. SCORE 1 ) 



III. THE RANKING PROCESS 



A CONSTRUCT RANKING MATRIX FORMAT 

1. SPECIFY ROW TITLES 

2. SPECIFY COLUMN TITLES 

3. LOCATE MATRIX CELLS (ONE CELL PER PERFORMANCE MEASURE) 



B. IDENTIFY CANDIDATE OCCUPATIONS FOR PROGRAM PLANNING 



Figure 1. Steps In Implementing the method 
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C. IDENTIFY ONGOING PROGRAMS FOR PROGRAM EVALUATION 



D. WEIGHT PLANNING OR EVALUATION COMPONENTS FOR RELATIVE IMPORTANCE (A VALUING 



E. WEIGHT INFORMATION CATEGORIES (A VALUING PROCEDURE) 



F. WEIGHT PERFORMANCE INDICATORS (A VALUING PROCEDURE) 



G WEIGHT PERFORMANCE MEASURES (A VALUING PROCEDURE) 



H. ASSIGN NORMALIZED PERFORMANCE MEASURE SCORES TO CELLS OF MATRIX 



I. ADJUST CELL SCORES FOR DATA INTEGRITY IF RANGE-NORMALIZING PROCEDURE HAS BEEN USED 



J. RECOMPUTE NORMALIZED PERFORMANCE MEASURE SCORES ((WEIGHT FOR PERFORMANCE MEA- 
SURE) X (NORMALIZED SCORE ADJUSTED FOR UNADJUSTED FOR DATA INTEGRITY)) 



K. RANK CANDIDATE OCCUPATIONS OR ONGOING PROGRAMS 

1. TOTAL EACH ROW OF RECOMPUTED SCORES 

2. PLACE THESE ROW TOTALS IN DESCENDING ORDER IF RANKING CANDIDATES 

3. PLACE TOTALS IN ASCENDING ORDER IF RANKING ONGOING PROGRAMS 



Rgure 1. Continued 



The design of the method was influenced by studies conducted by National Center staff and 
others (e.g.. Young 1973; Drewes and Katz 1975; Starr et al. 1 980; National Occupational Information 
Coordinating Committee 1 981; Starr et al. 1 981; Franchak 1 983). These studies suggest that vocational 
education administrators do support planning and evaluation so that programs will better meet 
employment needs. However, uncertainty still exists about what kinds of data are most appropriate to 
use and how best to apply the Imperfect data that are available. 

Intended Audience 

This publication has been prepared primarily fa local- and state-level vocational education 
administrators, planners, and evaluators who must make program planning and evaluation findings 
and decisions. It is also directed to management information system administrators who seek to 
improve the usefulness of this information specifically for program planning and evaluation purposes. 
The method described in this publication is designed mainly for specialized or area schools offering 
vocational education, comprehensive high schools and postsecondary institutions with sufficient 
instructional offerings to require a vocational administrator, and state vocational education 
agencies. 

Although the method has not been field-tested, an external panel did judgmentally assess it to 
suggest ways to improve its design and usefulness. The panel members found that it was conceptually 
sound and deserved consideration by vocational education administrators. 
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Chapter 2 
The method 

Chapter 2 provides a description of the method. Specifications that guided its design are 
presented first and are followed by a description of the method's components and procedures for 
implementing them. Exhibits and appendixes are used toenable the readerto visualize howplanning 
and evaluation information and data may be systematically selected and used with the method, how 
data are quantified, and how candidate occupations or ongoing vocational education programs 
are ranked. 

Design Specifications 

The six design specifications that guided the development of the method are as follows: 

• Produce findings enabling vocational education administrators to make defensible, data- 
based decisions for increasing vocational education relevance in meeting the employment 
needs of students, employers, and labor market areas. 

• Be applicable for both program planning and program evaluation. 

• Rely heavily on data that are selected and manipulated systematically. 

• Make explicit provisions for judging the importance and quality of data used with the 
method. 

• Be suitable for implementation within state educational agencies and secondary and 
postsecondary schools that deliver vocational education under quite diverse employment, 
education, economic, demographic, and social contexts and constraints. 

• Enable users to produce planning and evaluation findings based on different assumptions 
about present and anticipated employment and educational conditions. This capability is 
especially useful if planning and evaluation decisions are to be made by schools located in 
areas undergoing rapid or uncertain changes in demography, employment, or other condi- 
tions likely to affect the delivery of vocational education. Microcomputer technology will 
enhance efficiency in the implementation of this design specification. 

The vocational education literature was reviewed to determine if there are any vocational 
education program planning and evaluation methods that already meet these six design specifica- 
tions. None were found. Three methods were, however, identified that include approaches and 
techniques that could be extended, modified, or refined to meet the design specifications (Young 
1973; Starr, Merz, and Zahniser 1982; and Michigan Vocational Education Resource Center n.d.). The 



method described by the Michigan Vocational Education Resource Center appears to be a modifica- 
tion and application of program planning and evaluation techniques and approaches described by 
Young (1973) and by Starr. Merz, and Zahnlser ( 1982). Table 1 depicts how well these three methods 
meet the six design specifications. 



TABLE 1 



COMPARISON OF THREE METHODS 



Specification 




Method 






Young 


Michigan 
V-T 


Starr, Merz, 
and Zahnlser 


Designed specifically for relevance to 
employment needs 


- 


- 


- 


Applicable for program planning and 
evaluation 




Yes 


Yes 


Data selected and manipulated 
systematically 








Contains provisions for valuing data 
quality and Importance 








Designed for use under widest range of 
context conditions 




Yes 


Yes 


Designed to accommodate different 
assumptions about present and future 
employment and education conditions 
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Description of the dlethod 

The method includes three components: the information and data selection framework, the 
scoring process, and the ranking process. These components and their associated tasks are described 
next. The reader may find it useful to refer to figure 1 (chapter 1) to understand the method's 
components and their interrelationships. 

I. The Information and 
Data Selection Framework 
(Steps l-A through l-E) 



A framework has been developed to enable users to select information and data systematically 
for use with the program planning or evaluation method. This framework consists of five sequentially 
interrelated elements labeled as follows: 

1. Purpose and problem 

2. Planning and evaluation components 

3. information categories 

4. Performance indicators 

5. Performance measures 

The following is a brief overview of the logic underlying the framework's development. 

The method h used for increasing the relevance of vocational education programming in 
meeting employment needs. Its purpose requires users to be specific about the problem or need that 
requires implementing the method and to analyze and quantify those aspects of the employment 
and vocational education contexts and conditions related to the planning or evaluation problem. To 
simplify these requirements, an information and data selection framework has been developed. 

A program planning or evaluation problem can, of course, be stated concisely or in detail, and this 
section of the publication describes the content that might be useful in both concise and comprehen- 
sive problem descriptions. A problem description should provide background information useful in 
identifying the data most applicable to the planning or evaluation problem. 

This publication also provides a four-step procedure for arriving at the data that will be used to 
compare candidate occupations being considered for program development and implementation 
or to compare ongoing programs to determine which most need improvement or should be con- 
sidered for termination. This four-step process considers the employment and education contexts and 
conditions that affect the relevance of candidate occupations or ongoing programs in meeting the 
employment needs of students, employers, and labor market areas. 
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Mien doing program planning and evaluation it is important to use information and data that 
describe ( 1 ) the employment context, (2) vocational education processes, ( 3) vocational education 
outputs, (4) vocational education outcomes, and (5) benefits from vocational education. Thus, the 
information and data selection framework consists of these five components and associated ele- 
ments and subelements. 

The first step requires users to select those components to be used in the information and data 
selection framework. In most instances, all five components will be used. If no information is available 
for a component, it will not be used. 



The next three steps of the framework help establish the best available quantitative measures for 
each selected planning and evaluation component. These steps involve specifying (1) information 
categories for each component (e.g., employment supply and demand and employment conditions 
for the employment context component); (2) one or more performance indicatorsforeach information 
category (e.g., the extent to which current demand exceeds available supply for the supply and 
demand category); and (3) performance measures, which are indicators transformed into a quantifi- 
able format (e.g., whether the average annual number of Job openings exceeds the number of 
completers from all public and private schools in the state offering training for this occupation as a 
measure of the demand and supply indicator). 

Thus, users can select planning and evaluation data that have been logically derived from a 
presenting problem. Using this framework, therefore, provides a tool for systematically obtaining data 
for comparing candidate occupations and ongoing programs. Exhibit 1 depicts the location of the 
five elements in the information and data selection framework. 

Each element includes other subelements that make up a complete information and data 
selection framework. These will be depicted in other exhibits as each element in the framework is 
discussed. 



Framework Element 1— Purpose and 
Problem (Steps I-A1 and I-A2) 

As a first step in implementing the framework, users are encouraged to prepare a description of 
thespecific problem that precipitated the need to increase the relevance of vocational education in 
meeting employment needs. A description of the program planning or evaluation problem providesa 
useful foundation for the following: 

• Determining the criteria (performances) that should be used to compare candidate occu- 
pations or ongoing vocational education instructional programs 

• Selecting criterion measures (performance measures) that are appropriate and meaningful 
with respect to the planning or evaluation problem 

Users may want to consider the following when describing a planning or evaluation problem: 

• Explaining why administrators or planners in the secondary school, postsecondary institution, 
or state office believe it is necessary to increase overall vocational education program 
relevance to meet employment needs 
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EXHIBIT 1 

INFORMATION SELECTION FRAMEWORK 
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• Describing economic, social, educational, and demographic conditions within the primary 
labor market area served by the school system that should be considered when attempting 
to increase the relevance of vocational education to meet employment needs 

• Summarizing institutional resources, capabilities, and constraints that are likely to effect a 
change in the number of vocational education offerings currently being offered 

Formulating a context description (sometimes called an environmental scan) can be helpful in 
describing demographic, social, economic, employment, and educational conditions that should be 
considered when dealing with a particular planning or evaluation problem. Chapter 3 provides an 
example of a context description. 1 



'This context description is a modification of one found in Starr. Merz, and Zahniser ( 1 982, pp. 1 6-31 ). 
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An example of a program planning need would be deciding on the most relevant vocational 
education programs for skilled workers who have lost their jobs because of technological changes in 
production methods or because thelrfirms are moving to new locations end the present labor market 
area is experiencing very high unemployment. Program evaluation needs Include deciding which 
ongoing vocational education programs continue to be most and least relevant for high school 
students entering an expanding employment market In which fewer but better paying jobs are 
available in Industrial occupations and many morebut lower paying jobs are being created In service 
occupations. 

In both examples, the employment needs of students, employers, and labor market areas are 
quite different. Therefore, the criteria (performances) and criterion (performance) measures used to 

C Z™?°!f? candidate occupations and ongoing vocational education offerings will likely be some- 
what different. 

Framework Element 2-Planning and 
Evaluation Components (Step l-B) 

The second element in the framework consists of the following five program components: 2 

• Employmentcontext Includes the educational, employment, economic, and demographic 
condl.ions and constraints within the labor market areas being served by a school's voca- 
tional education program that affect, or are likely to affect, the relevance of vocational 
offerings in meeting employment needs. 

• VocaHonal education procesm are student and Instructional characteristics that affect, or 
are likely to affect, vocational program operation and the success of these offerings in 
meeting employment needs. 

• Vocational education outputs are the number of persons who graduate or complete pro- 
grams and achieve their skill levels. 

• Vocational education outcomes apply to the employment-related consequences of voca- 
tional education (e.g., training-related placement rates and job-search time). 

• Benefits from vocational education are the contributions of vocational education to indi- 
viduals, the economy, and society. 

These five program planning and evaluation components are used in the framework to help 
derive and specify hformation categories. Exhibit 2 depicts the placement of the five program 
planning and evaluation components within the framework 



'Ibid., chapter 2. 



10 



21 



EXHIBIT 2 

PLANNING AND EVALUATION COMPONENTS 
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Framework Element 3— Information 
Categories (Step l-C) 

The third element of the framework consists of information categories for each of the five program 
planning and evaluation components. Just as these components serve as an organizing framework 
for deriving and specifying information categories, the latter are used to select the performance 
indicators (Step l-D) that measure if, or the extent to which, candidate occupations or ongoing 
vocational education offerings meet employment needs. 

Information categories are best understood by examining some examples. Examples for each of 
the five program planning and evaluation components are found in exhibit 3. An extended list of 
information categories is found in appendix A Users of the method may choose from among the 
information categories listed in exhibit 3 and appendix A, modify the categories, or formulate their 
own. 

Using the example of program planning for displaced workers presented under element 1 of the 
framework and the information categories listed in exhibit 3, many users of the method may consider 
job opportunities as being more important than employment conditions. The reverse will probably 
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EXHIBIT 3 



EXAMPLES OF INFORMATION CATEGORIES FOR THE 
FIVE PLANNING AND EVALUATION COMPONENTS 
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hold true if secondary-level students will be entering an emerging service-oriented local economy as 
described In element 1 The way In which the program planning and evaluation method deals with 
differences In importance that users assign to Information categories Is explained In thlschapter under 
the ranking process. 

Framework Element 4— Performance 
Indicators (Step l-D) 

One or more performance Indicators must be assigned toeach Information category selected for 
use. Performance indicators can perhaps be best understood by studying examples of them in 
relationship to Information categories. Some examples can be found In exhibit 4. An extended list 
appears In appendix A Users of the method may choose from among the performance indicators 
listed in exhibit 4 and appendix A, modify the categories, or formulate their own. 

Formulating the best and the optimum number of performance Indicators will require experience. 
Like information categories, the most pertinent performance Indicators will be highly dependent on 
the previously developed description of the problem and will also be constrained by the availability of 
timely and pertinent data for measuring performance achievement. 
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Performance indicators, like information categories, may be considered by a user as having 
different levels of Importance. The procedures for dealing with the relative importance of performance 
indicators is explained in this chapter under the ranking process. 



EXHIBIT 4 

EXAMPLES OF PERFORMANCE INDICATORS 
FOR SELECTED INFORMATION CATEGORIES 
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Framework Element 5— Performance 
Measures (Step l-E) 

One or more performance measures may be associated with each performance Indicator. The 
application of multiple performance measures In scoring and ranking candidate occupations or 
ongoing vocational education offerings Is also explained in this chapter under the scoring and 
ranking processes. 

Performance measure data can take a variety of forms (e.g.. numbers, ratios, trends, discriminat- 
ing values, or ranks). There are no specific rules for establishing the best or optimum number of 
performance measures for any set of performance Indicators. In general. It should be easier to select 
appropriate performance measures where an educational agency has a research department or 
well-developed management information system, and when users gain experience In Implementing 
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and operating the method and in gauging the credibility, availability, and stability of various perfor- 
mance measures. 

Exhibit 5 gives examples of performance measures for selected performance indicators. An 
extended setcan be found In appendix A Sources for performance measures are found Inappendix B. 
Users may choose or modify these examples or formulate their own. To give the reader a clearer 
understanding of a total framework, appendix A contains the five elements of the Information 
selection framework. 

Rrst-tlme users of the method may find that particular performance measures deemed highly 
desirable are simply unavailable when they are needed (e.g.. measures dealing with employer 
satisfaction with trainee performance or the dropout rate from programs being evaluated). However, 
an awareness that certain data would be very useful for program planning or program evaluation 
may stimulate the collection of such data for use In subsequent Implementations. 



EXHIBIT 5 

EXAMPLES OF PERFORMANCE MEASURES 
FOR SELECTED PERFORMANCE INDICATORS 
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II. The Scoring Process 
(Steps ll-A and ll-B) 

Selecting a Normalizing 
Procedure (Step ll-A) 

Just as scores on dissimilar tests can be compared using centiles, quartiles, stanines, and the like, 
some way must be found to compare seemingly dissimilar performance measure data. Such compar- 
isons can be accomplished by normalizing performance measure data. 

This can be done in at least two ways. Each performance measure can be scored on whether or 
not a particular performance level has been achieved. If this normalizing procedure is selected, each 
performance measure will be assigned a pass or fall score (i.e., a performance measure score of one to 
zero). Performance measures can also be scored on the relative extent to which performance has 
been achieved. If this procedure for normalizing is selected, each performance measure will have a 
score that falls somewhere within the same common range of scores (e.g., a range of one to three, a 
range of one to five). 

Both procedures are appiicable for use with the method. For convenience, the first will be 
labeled the pass/fail normalizing procedure and the second will be labeled the range normalizing 
procedure. 

The range normalizing procedure may be easier to implement than the pass/fail procedure by 
first-time method users because of the inherent difficulties of establishing and justifying levels of 
performance that discriminate between passing and failing. Starr, Merz, and Zahniser (1982) discuss 
examples from Young ( 1 973) and from their own work cf applying the two types of normalizing procedures 
(although they label them differently). 

Preparing Performance Measure Statements for 
Compatibility With a Normalizing Procedure (Step ll-B) 



Exhibits 6 and 7 provide examples of both procedures for scoring performance measures. Exhibit 6 
contains examples of performance measures formatted for use with the pass/fail procedure, whereas 
exhibit 7 contains examples for use with the range procedure. More extensive examples of formatted 
performance measures as well as performance measure scores using the two normalizing procedures 
are found in appendix A 
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EXHIBIT 6 



EXAMPLES OF PERFORMANCE MEASURES AND SCORES WHERE 
THE PASS/FAIL NORMALIZING PROCEDURE IS USED IN 
THE SCORING PROCESS 
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EXHIBIT 7 



EXAMPLES OF PERFORMANCE MEASURES AND SCORES WHERE THE 
RANGE NORMALIZING PROCEDURE IS USED WITH THE SCORING PROCESS 
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III. The Ranking Process 
(Steps lll-A through lll-K) 



The ranking process consists of severai procedures that culminate in ranking candidate occupa- 
tions or vocational education offerings. The rankings provide method users with a data-based and 
rationalized foundation for making decisions about which instructional programs to implement and 
which ongoing offerings to improve or terminate in order to increase totai vocational education 
program relevance in meeting employment needs. 

In implementing this process, it is necessary to construct a ranking process matrix (Step lll-A). 
Exhibit 8 depicts the row and column format for the matrix. 



Identifying Candidate Occupations or Vocational 
Education Offerings (Steps lll-B or lll-C) 



The program planning and evaluation method requires users to identify a set of candidate 
occupations that should be considered for program development and implementation or a set of 
ongoing vocational education instructional programs to be evaluated for program improvement or 
termination. 



Users can consult several sources, including trade associations, unions, professional groups, the 
state's employment services agency, and the local advisory committee for vocational education, to 
obtain a list of candidate occupations (Step lll-B). These sources could be supplied with selection 
criteria to aid them in recommending candidates. Sample selection criteria might inciude that (1 ) a 
candidate occupation require formal training (but iess than a college degree) by persons seeking 
entry-level employment and (2) a significant number of people are likely to be employed in the 
candidate occupation in various state or local iabor market areas. 

Selecting ongoing vocational education programs or program clusters for evaluation (Step lil-C) 
can be done in several ways. An administrator could select a sampie (e.g., 20 percent of all programs) 
and evaluate them once each year, or evaluate specific programs or program clusters if local 
employers or other groups are concerned about whether particular instructional programs meet 
employment needs of students, employers, or the local labor market area. 



Procedures for Valuing 
(Steps lll-D through lll-G) 



Since the method is vaiue-driven, users must carry out a set of procedures for (1 ) weighting the 
relative importance of planning and evaluation components, information categories, performance 
indicators, and performance measures and (2) modifying normalized performance measure scores to 
reflect these weightings. Different sets of weightings are likeiy to resuit in different rankings for candi- 
date occupation or ongoing vocational education programs. The procedures for weighting orvaiuing 
are described next. 
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EXHIBITS 


















A SAMPLE RANKING PROCESS MATRIX 
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NOTE: To save space, the output component has been omitted in exhibits 8-12. 
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Weighting Planning and Evaluation Components (Step lll-D). 



One way to approach this task is to assume that the sum of the weightings for the set of 
components must equal 100 points. If all components are judged equally important each compo- 
nent weighting equals 100 divided by the number of components. If all 5 components are used when 
implementing the method, each receives 20 points (I.e.. 100/5 = 20), If some planning and evaluation 
components are judged more important than others, then a proportion of the 100-point total is 
assigned to each (e.g., 20 points, 30 points, 15 points, 10 points, and 25 points, respectively). Exhibit 9 
depicts this weighting scheme. 

Weighting Information Categories for Importance (Step lll-E). 

One or more information categories will be associated with each planning and evaluation 
component. If there is only one information category within a component, that category receives the 
same weight as the component. If there are two or more, the following weighting scheme is recom- 
mended. If all categories are considered equally important, the weight assigned toeach is derived by 
dividing the component weight by the number of information categories (e.g., if the component 
weight equals 30 points and there are 3 information categories, each receives 1 0 points). On the other 
hand, if a user decides that the categories are not equally Important, the categories would be 
assigned a proportion of the points assigned to their respective component (e.g., if the component 
weight equals 30 points and there are 3 information categories, the category weights might be 10 
points, 15 points, and 5 points or perhaps 15 points, 7.5 points, and 7.5 points). This weighting scheme is 
depicted in exhibit 10. 

Weighting Performance Indicators (Step Ill-F). 

Each information categojy will have one or more associated performance Indicators. If there is 
only one, it receives the same weight as the category. If there are two or more indicators within a 
category, the following weighting scheme is recommended. If all performance indicators within the 
information category are considered equally important, the weight assigned to each is derived by 
dividing the category weight by the number of performance indicators (e.g., if the information 
category weight equals 10 points and there are 2 performance indicators, each indicator receives 5 
points). On the other hand, if a user decides that the indicators are not equally important they receive 
a proportion of the points assigned to their respective category (e.g., if the category weight equals 1 0 
points and there are 2 performance indicators, the indicator weights might be 8 points and 2 points or 
perhaps 4 points and 6 points). This weighting scheme is depicted in exhibit 11, 

Weighting Performance Measures (Step lll-G). 

Each performance indicator will have one or more associated performance measures. If there is 
only one, it receives the same weight as the indicator. If there are two or more, the following weighting 
scheme is recommended. If all measures are considered equally important, the weight assigned to 
each is derived by dividing the performance indicator weight by the number of performance mea- 
sures (e.g., if the performance indicator weight equals 15 points and there are 2 performance 
measures, each receives 7.5 points). If the 2 measures are valued differently, their combined point total 
must equal the performance indicator weighting (e.g., if the performance indicator weight is 15, the 2 
measures might receive 5 points and 10points, respectively). Exhibit 12 depicts this weighting scheme. 



EXHIBIT 9 

WEIGHTING PLANNING AND EVALUATION COMPONENTS 
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Indicators (2) (1) (1) (1) (1) (1) (1) (1) (1) 
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EXHIBIT 10 
WEIGHTING INFORMATION CATEGOWED 
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EXHIBIT 11 

WEIGHTING PERFORMANCE INDICATORS 
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EXHIBIT 12 
WEIGHTING PERFORMANCE MEASURES 
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Assigning Normalized Performance 
Measure Scores to Cells (Step lll-H) 



For each candidate occupation or ongoing vocational education program, there will be one 
normalized performance measure score per performance measure. The ranking process matrix con- 
tains one cell per performance measure score. If the pass/fail normalizing procedure has been used 
with tha method, each matrix cell will be assigned a score of zero or one. If the range normalizing 
procedure has been used, each cell will receive a score from one to the specified upper end of the 
measurement range (e.g., three, four, or five). 

Adjusting Normalized Performance Measure 
Scores for Data Quality (Step lil-l) 

Users may consider data used to produce normalized performance measure scores completely 
acceptable. However, this is sometimes not the case. There are times when employment, education, 
demographic, and economic data may be considered less than completely acceptable because of 
questionable credibility, reliability, or accuracy. In such cases, the data quality can be considered as 
being impaired. 

Adjusting normalized performance measure scores for data quality is only appropriate when the 
range normalizing procedure is used. When the pass/fail normalizing procedure is used, no corrections 
for data quality should be made. When performance measure scores have been generated using the 
pass/fail normalizing procedure, the quality of performance data should be considered fully accept- 
able. Scores represent performances that either have or have not been achieved; there is no range of 
scores. However, if corrections for data quality were permitted, pass/fail values would, in effect, be 
converted to a range of scores. 

Applying Weightings to Account for Differences in Data Quality. 

if the range normalizing procedure has been used and the option to adjust normalized scores for 
data quality is elected, users may assign a value of parity or equivalence to the scores when there are 
no serious doubts about the quality of the data used to generate them and reduce them by fixed 
amounts depending on their adjudged degree of impairment. An example follows. Reduce obtained 
normalized performance measure scores for data quality— 

• by 0. 1 if performance measure data reliability, accuracy, or credibility seems to be only mildly 
suspect; 

• by 0.2 if performance measure data reliability, accuracy, or credibility seems to be moder- 
ately suspect; 

• by 0.3 if performance measure data reliability, accuracy, or credibility seems to be marginally 
passable. 

It would not be advisable to adjust normalized performance measure scores for data quality 
unless users are knowledgeable about the particular data being used for scoring. Data should not be 
used with the method if they are believed to be really "bad." Exhibit 13 depicts normalized perfor- 
mance measure scores adjusted for data quality. 
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EXHIBIT 13 

ADJUSTING NORMALIZED PERFORMANCE MEASURE SCORES FOR DATA QUALITY 



Planning and 

Evaluation 

Component) 


Context 




Process 










Outcome 


Benefits 


Wwigni iQf 
Importance 


40.0 




30.0 










15.0 


15.0 


information 
Categories 


Job 
Opportunities 


Employment 
Conditions 


Staffing 


Dollars 


Facilities 


Student 
interest 


Training 
Availa- 
bility 


Placement 


Economic 
Benefits 


Weight for 
importance 


25.0 


15.0 


5.0 


10.0 


5.0 


5.0 


5.0 


15.0 


15.0 


Performance 
Indicators 


1 2 
(2) 


3 

d) 


4 

d) 


5 

(1) 


6 

(D 


7 

(1) 


8 

(1) 


9 

(1) 


10 

(D 


Weight for 
importance 


12.5 


12.5 


15.0 


5.0 


10.0 


5.0 


5.0 


5.0 


15.0 


15.0 


Performance 
Measures 


a 


a 


a 


a 


a 


a 


a 


a 


a 


a 


renointunvv 

Woight 


12.5 


12.5 


15,0 


5.0 


10.0 


5.0 


5.0 


5.0 


15.0 


15.0 


Machinist 
Technician 
(normalized) 




















Sum 


2.0 


2.0 


2.0 


3.0 


1.0 


2.0 


2.0 


2.0 


3.0 


2.0 


(adjusted) 


1.8 


2.0 


2.0 


3.0 


1.0 


2.0 


1.9 


2.0 


3.0 


2.0 


(recomputed) 


22.5 


25.0 


30.0 


15.0 


10.0 


10.0 


9.5 


10.0 


45.0 


30.0 207.0 



Recomputing Normalized Performance 

Measure Scores for Weightings of importance (Step IlkJ) 



The normalized performance measure scores, adjusted or unadjusted for data quality, must now 
be recomputed to reflect the values (weightings of importance) assigned to planning and evaluation 
components, information categories, performance indicators, and performance measures. The sim- 
plest way is to multiply each normalized performance measure score, adjusted or unadjusted for data 
quality, by its corresponding performance measure weight. The recomputation process is depicted in 
exhibit 14. 

Ranking Candidate Occupations or 
Ongoing Programs (Step HI-K1 and III-K2) 

The last steps in the process include ranking candidate occupations or ongoing vocational 
offerings. 

Ranking Candidate Occupations. 

Each row of recomputed cell scores should be summed (Step III-K2), which produces a total row 
score for each candidate occupation. The totals for all candidate occupations should be placed in 
descending order (Step III-K2). Those occupations with the highest totals would be selected as being 
most relevant for meeting employment needs and thus as most appropriate for program develop- 
ment and implementation. This ranking procedure is depicted in exhibits 17 and 18 in chapter 3. 

Ranking Ongoing Programs. 

Each row of recomputed cell scores should be summed (Step III-K1 ), producing a total score for 
each vocational education program being evaluated for relevance in meeting employment needs. 
The totals for all these programs should be placed in ascending order (Step III-K3). The programs with 
the lowest totals would be considered least relevant for meeting employment needs and therefore 
most in need of program improvement or consideration for termination. 

Increased Ranking Process Efficiency 
by Using a microcomputer 

The ranking process can be made more efficient by using a microcomputer and a spreadsheet 
program, which creates a matrix on a microcomputer monitor. The matrix consists of rows, columns, 
and cells. Users fill these with labels, numbers, or text. 

Planning and evaluation components, information categories, performance indicators, and 
performance measures as well as the weights for the importance of each element can be entered on 
the monitor screen in the columns of the matrix. Candidate occupations or ongoing vocational 
education offerings can be entered in the rows of the matrix. Normalized, adjusted, and recomputed 
performance measure scores can be entered in the cells that correspond to their proper candidate 
and performance measure intersections. 
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EXHIBIT 14 

RECOMPUTING NORMALIZED PERFORMANCE MEASURE SCORES AFTER ADJUSTMENTS FOR DATA QUALITY 



Planning and 

Evaluation 

Components 


Context 




nwwwM 










Outcome 


Benefits 


Weight for 
Importance 


40.0 




30.0 










15.0 


15.0 


Information 
Categories 


Job 
Opportunities 


Emolovment 
Conditions 




L/V/IIQlS 


Facilities 


Ml, 1 A 

Student 
Interest 


Training 
Availa- 
bility 


Placement 


Economic 
Benefits 


Weight for 
Importance 


25.0 


15.0 


5.0 


10.0 


5.0 


5.0 


5.0 


15.0 


15.0 


Performance 
Indicators 


1 2 
(2) 


3 

d) 


4 

(1) 


5 

d) 


6 

d) 


7 

d) 


8 

(D 


9 

d) 


10 

(1) 



Weight for 

Importance 12.5 12.5 15.0 5.0 10.0 5.0 5.0 5.0 15.0 15.0 



Performance 
Measures 



Sum 



Performance 

Weight 12.5 12.5 15.0 5.0 10.0 5.0 5.0 5.0 15.0 15.0 

Machinist 
Technician 
(normalized) 2.0 

(adjusted) 1.8 

Oecomputed)* 1 22.5 25.0 30.0 150 10.0 10,0 9 .5 10.0 45.0 30,0 207.0 1 

•Recomputed cell score = (performance weight X (adjusted cell score). In the event adjustments for data quality are not made the 
recomputed cell score = (performance weight) X (normalized performance measure score). 



2.0 


2.0 


3.0 


1.0 


2.0 


2.0 


2.0 


3.0 


2.0 


2.0 


2.0 


3.0 


1.0 


2.0 


1.9 


2.0 


3.0 


2.0 


25.0 


30.0 


15.0 


10.0 


10.0 


9.5 


10.0 


45.0 


30.0 



Algebraic formulas and statistical functions can be assigned to cells to ( 1 ) automatically recom- 
pute all normalized performance measure scores whenever levels of importance assigned to planning 
and evaluation components, information categories, performance indicators, or performance mea- 
sures change and (2) compute new row totals for all candidate occupations or ongoing programs. 
These formulas and functions can recalculate the contents of specific cells, rows of cells, or columns of 
cells. Recalculated totals can be further directed by appropriate spreadsheet program commands to 
be reordered in ascending or descending order. 

A spreadsheet program helps with the scoring process especially when users are uncertain or differ 
in opinion about the weightings of importance that should be assigned to planning and evaluation 
components, information categories, performance indicators, and performance measures. Users may 
also want to reweight (revalue) these elements based on alternate assumptions about employment 
needs as these may exist in the next 5 or 10 years. Using a spreadsheet program and microcomputer 
can speed up the recalculation process required by any revaluing. Chapter 3 discusses applying a 
spreadsheet program to the ranking process. 



Chapter 3 
implementing the method— 
fin Enemple 

Chapter 3 illustrates use of the method fordoing program planning at the local level. The example 
uses the range normalizing procedure. An example for evaluating ongoing instructional programs by 
using the pass/fail normalizing procedure is not included to avoid presenting more detail than is 
necessary. 

The Site 



The planning and evaluation method will be implemented at Grand Community College. The 
college, one of the state's many publicly supported postsecondary institutions, is located in Expansion 
City, the major population center in Grand County. About 80 percent of the college's students are 
enrolled in vocational education programs. 

This institution regularly offers several technology programs. Graduates of these programs receive 
an associate degree. The college also offers a variety cf vocational education programs that lead to a 
certificate of completion. Some of these programs are offered on a regular basis, whereas others are 
offered only as the need arises (e.g., in response to a firm's request to prepare, retrain, or upgrade 
employees). 

Grand Community College receives about 45 percent of its operating funds from the county, 
about 30 percent from students' tuition, and about 25 percent from the State Board for Postsecondary 
Education. The bulk of the college's funding is from local sources; therefore, the community expects 
that vocational-technical programs will mainly serve the Grand labor market area. 

Several recent actions by the governor's office and the state legislature have been directed 
toward improving employment training responsiveness in meeting state and local employment and 
economic needs. Members of the state legislature are on record as supporting postsecondary 
education institutions providing the kinds of trained workers that will induce manufacturing and 
service firms to maintain, relocate, or begin operations in the state. The State Board for Postsecondary 
Education has responded to the legislative concern by requesting that all publicly supported com- 
munity colleges and technical institutes determine what newvocationaleducation programs should 
be offered in their labor market areas to increase responsiveness to economic development needs. 

These events led the governing board of Grand Community College to urge the school's president 
to determine what new vocational education programs will best meet local resident and potential 
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employers needs. The president notified the dean of occupational education about the governing 
board's concerns and the need for immediate action. Subsequently, the dean selected a program 
planning team that included college staff and persons from the community (e.g., employers and 
members of the local economic development council). 

At its first meeting, the team reviewed the circumstances leading to its appointment. The dean of 
occupational education, chairperson of the team, pointed out that Grand Community College has 
alwaysengaged in formal planning before implementing new occupational education programs. For 
example, occupational education staff always consult with a local advisory committee composed of 
representatives of local area firms before implementing new occupational programs. In addition, the 
staff have conducted numerous surveys of employers and adults to determine what kinds of occupa- 
tional education programs are desired in the local labor market area. 

Some of the team's members voiced concerns about relying so heavily on advice from advisory 
committees and data from employment surveys because employers' expressed opinions of needs for 
trained workers might be overly self-serving or might be based on different sets of assumptions about 
the economy and the availability of adequately t, ined workers. Since team members would not be 
aware of these assumptions, the advice received and the data obtained could not be properly 
interpreted. Other team members stated that previous program planning methods were more than 
acceptable. 

In view of these contrasting views, the dean suggested that the team consider using a program 
planning and evaluation method described in a publication issued by the National Center for 
Research in Vocational Education at The Ohio State University, if they were not satisfied with that 
method, the team could fail back on surveys or advice from an advisory committee. The team agreed 
to review documentation describing the method. 

After reading the documentation, some team members voiced a concern that it was untested, 
complex, and potentially difficult to implement. They did agree, however, that the method had some 
noteworthy features. In the end, the team elected to adopt the method described in the National 
Center publication. 

At a subsequent planning team meeting, the dean appointed a subcommittee to formulate a 
more complete description of the planning problem given to them. A context description, which is 
suggested for use with the method, was not available. Therefore, the subcommittee decided to 
develop one. A copy of the Grand Labor Market Area Context Description is reproduced at the end of 
this chapter. 

The team's next problem was to identify a set of candidate occupations to be considered for 
program development and implementation. It was decided that a group of persons not connected 
with the team should be appointed for this purpose. The dean contacted the director of the local 
economic development council, the executive director of the local chamber of commerce, and three 
persons involved in professional and trade associations and requested that they serve as an ad hoc 
committee. 

The ad hoc committee members were informed of their responsibility for nominating candidate 
occupations. These persons were further informed that the list of candidate occupations should 
include those (1) found in at least four fields of interest to vocational education (e.g., health, 
commerce, construction, and technology), (2) found in both the manufacturing and service sectors, 
and (3) determined to be either new or emerging in the local labor market area or to have potential 
economic development consequences (i.e., the presence of adequate numbers of trained workers in 
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these occupations might induce firms to locate in this labor market area or keep local firms from 
relocating a closing). 

The president of the community college added one more criterion for selecting candidate 
occupations: Some must be occupations for which disadvantaged students can be trained (even if 
the college must provide compensatory education courses). The ad hoc committee members agreed 
that this issue had to be addressed, given the comments about disadvantaged groups mentioned in 
the context description and the fact that Grand Community College is committed to serving the 
educationally disadvantaged. They completed their deliberations and presented the planning team 
with a list of candidate occupations (exhibit 15). 



EXHIBIT 15 
LIST OF CANDIDATE OCCUPATIONS 

1. Machinist technician 

2. Automotive technician 

3. Data processing machine repairer 

4. Word processor operator 

5. Lab technician (medical) 

6. Salesperson (retail) 

7. Electronic technician 



The planning team then turned its attention to determining systematically what kinds of informa- 
tion to use with the method. A subcommittee was appointed by the dean to carry out this task. They 
began by studying examples of information categories, performance indicators, and performance 
measures found in appendix A of the publication describing the method. The subcommittee then met 
with the college's management information system director and subsequently with staff from the local 
office of the state employment services agency to ascertain the availability and quality of various 
education and employment data. 

The subcommittee members learned that some desired data required to measure the relevance 
of candidate occupations in meeting employment needs could not be secured from the state 
employment services agency, the state departments of education or postsecondary education, or 
the college's management information system unit. They also found out that not all available educa- 
tion, employment, and labor market information was of equal quality. 

The subcommittee reviewed the planning problem, the context description, and the data con- 
straints described in the meeting with the management information director and the staff from the 
employment services agency. They were able to agree about the specific planning and evaluation 
components, information categories, performance indicators, performance measures, and the nor- 
malizing procedure that should be used with the method. Exhibit 16 contains the subcommittee's 
recommendations as approved by the team. 




EXHIBIT 16 



LIST OF INFORMATION CATEGORIES, PERFORMANCE INDICATORS, AND PERFORMANCE MEASURES (AND SCORES) 



Information Categories 



Performance Indicators 



Performance Measures (and Scores) 



Job Opportunities 



(1 ) The extent of the growth rate of 
this occupation 



(2) The extent to which future 
demand for workers is great 
enough to consider implement- 
ing a program 



(a) 3 If this occupation Is among the 10 fastest-growing 
occupations in this labor market area 

2 If this occupation is not among the 10 fastest- 
growing ones, but is exhibiting growth 

1 If this occupation's projected rate of growth is 
expected to be constant or else decline 

(a) 3 If the average annual number of job openings Is 

greater than the number of completers in the 
past 12 months from all known programs in this 
labor market area preparing persons for this 
occupation 

2 If the average annual number of job openings Is 
less than or equal to the number of completers In 
the past 12 months from all known programs In 
this labor market area preparing persons for this 
occupation 

1 If the number of completers is unknown 

(b) 3 If national growth prospects for this occupation 

are considered above average 

2 If national growth prospects for this occupation 
are considered average or below average 

1 If growth prospects are unreported 



FRir 



4o 



EXHIBIT 16— Conf/nt/ed 



Information Categories Performance Indicators Performance Measures (and Scores) 



Conditions of 
Employment 



Staffing 



Dollars 



(3) The extent to which employers 
practice affirmative action in 
hiring persons for this 
occupation 



(4) The extent to which staff can be 
made available to teach the 
skills required in preparing 
persons for this occupation 



(5) The extent to which costs are 
acceptable 



(a) 3 If there is evidence that most employers hiring 
persons for this occupation do not discriminate 
against minorities or the economically 
disadvantaged 

2 If there is evidence that most employers do 
discriminate 

1 1f affirmative action practices are unknown 

(a) 3 If there are staff who can be assigned without 
compromising any ongoing programs 

2 If there are staff who can be assigned but it 
means that 1 or more programs will be affected 
to some extent 

1 If no staff are currently available and recruitment 
is uncertain 

(a) 3 If the recurrent cost is likely to be less than the 
median of recurrent costs for continuing pro- 
grams at this school 

2 If the recurrent cost is likely to equal or exceed 
the median of recurrent costs for continuing pro- 
grams at this school 

1 If the recurrent cost cannot be estimated at this 
time 
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EXHIBIT 16-Continued 



Information Categories 



Performance Indicators 



Performance Measures (and Scores) 



Facilities 



Student Interest 



(6) The extent to which space 
needed for instruction to train 
persons for this occupation is 
acceptable 



Placement 



(7) The extent to which there is likely 
to be sufficient enrollment to jus- 
tify implementing a program to 
prepare persons for this 
occupation 



(8) The extent to which program 
graduates find training-specific 
or training-related jobs 



(a) 3 If less space would be required than the median 
space used for other continuing programs in this 
school 

2 If space required would be equal to or greater 
than the median space used for other continuing 
programs 

1 If space required by the candidate cannot be 
estimated at this time 

(a) 3 If there is a trend toward increased enrollments 
statewide in programs to prepare persons for this 
occupation 

2 If enrollments statewide have remained essen- 
tially stable 

1 If enrollments are declining 

(a) 3 If the placement rate exceeds the state average 
for this type of program by +1 standard deviation 

2 If the placement rate for this type of program, 
statewide, is within +1 and -1 standard deviations 

1 If the placement rate for this type of program, 
statewide, is more than -1 standard deviation 
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EXHIBIT 16-Continued 



Information Categories 



Performance Indicators 



Availability of Training 
Opportunities 



(9) The extent to which training is 
offered by other schools in this 
labor market area 



Economic Benefits 



(10) The extent to which the avail- 
ability of a program to train per- 
sons for this occupation is likely 
to be an incentive in interesting 
firms to either relocate here or 
remain in this area 
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Performance Measures (and Scores) 



(a) 3 If there are no other schools in this area offering a 
program to prepare persons for this occupation 

2 If there is only 1 other school in this area offering 
training for this occupation 

1 If several schools in this area offer training for this 
occupation 

(a) 3 If the local economic development agency has 
received 5 or more inquiries about the availability 
of such training during the past 12 months 

2 If the local economic development agency has 
received from 1 to 4 inquiries during the past 12 
months 

1 If the local economic development agency has 
received no inquiries during the past 12 months 



ERIC 



The Ranking Process 



The team nominated the dean and the management information system unit director to imple- 
ment the ranking process. Available data were assembled and normalized scores were assigned to 
each performance measure. Since the range normalization procedure was being used, they decided 
to correct performance measure scoresfordata quality, which drew no adverse comments from other 
team members. 

Business and industry team members vehemently disagreed, however, with the members repre- 
senting the education community about the weightings of importance that the dean and the 
management information system director assigned to some of the planning and evaluation compo- 
nents. The business and industry representatives concluded that the context description pointed out 
the need to score the employment context component higher than the education-related compo- 
nents. The educators on the team held just the opposite point of view. 

After a lively debate, it was agreed that there should be two different sets of weightings of 
Importance for planning and evaluation components, that statements should be prepared describ- 
ing the rationale for the two different sets of weightings, and that the ranking process should be 
implemented twice to produce two rankings of the candidate occupations. The team agreed to 
provide the college's president with th is information and to let that individual decide which of the two 
rankings would be most acceptable. 

The National Center report indicated that a spreadsheet program could hasten the ranking 
process when more than one set of weightings was being considered. However, since there were only 
two rankings to be performed, manually computing rankings seemed more efficient than using a 
spreadsheet program, which takes time to set up. Members of the team were interested, however, in 
testing the efficiency of using a spreadsheet program because the president and the governing 
board would probably want to include their own assumptions and judgments about weightings for 
importance and adjustments for data quality. 

The director of the management information system unit brought in a spreadsheet program and a 
mterocomputer. set up the spreadsheet matrix to duplicate the elements of the ranking process 
matrix, and entered cell scores, formulas, and functions as appropriate. The director ran the spread- 
sheet program after inserting the two alternative sets of weightings. Exhibits 17 and 18 depict the 

resu ts. Note that in this situation, the two sets of weightings did alter the rankings somewhat. However, 
the top two occupations kept their rankings. Most team members saw the value in using a spread- 
sheet computer program to test the impact of different weighting scenarios. 

Grand Labor market Area ConteHt Description 

Context for Employment 

rnuE 0 ^' °1 e omrT \ unitV of 135 <°0° Persons, is the county seat of Grand County. Grand 
SS^^'^S!?;* * T8rrifiC and Moreso form the Grand *>ndaid metVopoSan 
S^SSSSS£?"i . S I™ 0 populQtion of approximately 315.000. Terrific and Moreso 
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EXHIBIT 17 

SPREADSHEET DETAILING COMPUTATIONS AND RANKINGS— SET 1 



Planning and 

Evaluation 

Components 


Context 




Process 










Outcome 


Benefits 


Weight for 
Importance 




40.0 




30.0 










15.0 


15.0 


Information 
Categories 


Job 
Opportunities 


employment 

Conditions 


Starling 


Dollars 


Facilities 


Student 
Interest 


Training 
Avail- 
ability 


Placement 


Economic 
Benefits 


Weight for 
Importance 




25.0 


15.0 


5.0 


10.0 


5.0 


5.0 


5.0 


15.0 


15.0 


Performance 
Indicators 


1 


2 

(2) 


3 

(1) 


4 

d) 


5 

(1) 


6 

(1) 


7 

(1) 


8 

(1) 


9 

(1) 


10 
(1) 


Weight for 
Importance 


12.5 


125 


15.0 


5.0 


10.0 


5.0 


5.0 


5.0 


15.0 


15.0 


Performance 
Measures 


a 


a 


a 


a 


a 


a 


a 


a 


a 


a 


Performance 
Weight 


12.5 


12.5 


15.0 


5.0 


10.0 


5.0 


50 




1 o.u 


I O.U 


Machinist 
Technician 
(normalized) 


2.0 


2.0 


2.0 


3.0 


1.0 


2.0 


2.0 


2.0 


3.0 


Sum 

2.0 


(adjusted) 


18 


2.0 


2.0 


3.0 


1.0 


2.0 


1.9 


2.0 


3.0 


2.0 


(recomputed)* 


22.5 


25.0 


30.0 


15.0 


10.0 


10.0 


9.5 


10.0 


45.0 


30.0 207.0 



•Recomputed cell score = (Performance weight X (adjusted cell score). In the event adjustments for data quality are not made 
the recomputed cell score = (performance weight) X (normalized performance measure score). 



EXHIBIT 1 7- Continued 



Automotive 
Technician 
\normonzeo; 


1.0 


2.0 


2.0 


3.0 


2.0 


1.0 


2.0 


2.0 


1.0 


2.0 




(adjusted) 


0.8 


2.0 


2.0 


3.0 


2.0 


1.0 


1.9 


2.0 


1.0 


2.0 




(recomputed) 


10.0 


25.0 


30.0 


15.0 


20.0 


5.0 


9.5 


10.0 


15.0 


30.0 


169.5 


Computer 
Programmer 
(normal izea) 


2.0 


3.0 


2.0 


3.0 


3.0 


1.0 


2.0 


3.0 


2.0 


2.0 




(adjusted) 


1.8 


3.0 


2.0 


3.0 


3.0 


1.0 


1.9 


3.0 


2.0 


2.0 




(recomputed) 


22.5 


37.5 


30.0 


15.0 


30.0 


5.0 


9.5 


15.0 


30.0 


30.0 


224.5 


Word 
Processor 
inormajzea) 


2.0 


2.0 


2.0 


1.0 


1.0 


2.0 


2.0 


1.0 


2.0 


1.0 




(adjusted) 


1.8 


2.0 


?0 


1.0 


1.0 


2.0 


1.9 


1.0 


2.0 


1.0 




(recomputed) 


22.5 


25.0 


30.0 


5.0 


10.0 


10.0 


9.5 


5.0 


30.0 


15.0 


162.0 


Medical Lab 
Technician 
(normalized) 


2.0 


2.0 


2.0 


2.0 


3.0 


2.0 


2.0 


2.0 


2.0 


2.0 




(adjusted) 


1.8 


2.0 


2.0 


2.0 


3.0 


2.0 


1.9 


2.0 


2.0 


2.0 




(recomputed) 


22.5 


25.0 


30.0 


10.0 


30.0 


10.0 


9.5 


10.0 


30.0 


30.0 


207.0 


Salesperson- 
retail 

(normalized) 


3.0 


2.0 


3.0 


2.0 


3.0 


2.0 


1.0 


2.0 


1.0 


1.0 




(adjusted) 


2.8 


2.0 


3.0 


2.0 


3.0 


2.0 


0.9 


2.0 


1.0 


1.0 




(recomputed) 


35.0 


25.0 


45.0 


10.0 


30.0 


10.0 


4.5 


10.0 


15.0 


15.0 


199.0 



EXHIBIT 1 7— Continued 



0 

ERIC 



Electronic 
Technician 
(normalized) 


2.0 


30 


2.0 


3.0 


2.0 


2.0 


3.0 


2.0 


3.0 


3.0 


(adjusted) 


1.8 


3.0 


2.0 


3.0 


2.0 


2.0 


2.9 


2.0 


3.0 


3.0 


(recomputed) 


22.5 


37.5 


30.0 


15.0 


20.0 


10.0 


14.5 


10.0 


45.0 


45.0 2 



Performance Indicators 

1. Extent of rate of growth 

2. Extent of future demand 

3. Extent of affirmative action 

4. Extent to which staff are 
available 

5. Acceptability of costs 



6. Extent to which space is 
acceptable 

7. Extent to which enrollments 
likely to be sufficient 

8. Extent to which training offered 
by other sources 

9. Extent of placement in occu- 
pation for which trained 

10. Extent of incentive for eco- 
nomic development purpose 



Performance Measures 



RANK 
ORDER 



SUM 



OCCUPATION 



a. Fastest growing 


1 


249.5 


Electronic technician 




2 


224.5 


Computer programmer 


a. Average annual openings 


3 


207.0 


Machinist technician 




4 


207.0 


Medical lab technician 


a. Evidence of nondiscrimination 


5 


199.5 


Salesperson (retail) 




6 


169.5 


Automotive technician 


a. Current and potential 


7 


162.0 


Word processor 



availability 

a. Comparison of recurrent costs 
with other programs, locally 

a. Comparison of space needs to 
other programs 

a. Enrollment trends, statewide 



a. Other schools in LMA offering 
training for occupation 

a. Comparison of placement rates 
for similar programs, statewide 

a. Number of inquiries about train- 
ing availability 
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EXHIBIT 18 

SPREADSHEET DETAILING OCCUPATIONS AND RANKINGS— SET 2 



Manning and 

Evaluation 

Components 


Context 




Process 










Outcome 


Benefits 


weignf ror 
Importance 




30.0 




40.0 










15.0 


15.0 


Information 
Categories 


Job 
Opportunities 


Employment 
Conditions 


Staffing 


Dollars 


Facilities 


Student 
Interest 


Training 
Avail- 
ability 


Placement 


Economic 
Benefits 


Weight for 
Importance 




20.0 


10.0 


5.0 


20.0 


5.0 


5.0 


5.0 


15.0 




Performance 
Indicators 


1 


2 

(2) 


3 

d) 


4 

d) 


5 

d) 


6 

d) 


7 

(1) 


8 

d) 


9 

(1) 


10 

(1) 


Weight for 
Importance 


10.0 


10.0 


10.0 


5.0 


20.0 


5.0 


5.0 


5.0 


15.0 


15.0 


Perfo'fnanfA 

* vl | V/ti 1 1 IUI IWv 

Measures 


a 


a 


a 


a 


a 


a 


a 


a 


a 


a 


Performance 
weignf 


10.0 


10.0 


10.0 


5.0 


20.0 


5.0 


5.0 


.5.0 


'i 5.0 


15.0 


Machinist 
Technician 
(normalized) 


2.0 


2.0 


2.0 


3.0 


1.0 


2.0 


2.0 


2.0 


3.0 


Sum 

2.0 


(adjusted) 


1.8 


2.0 


2.0 


3.0 


1.0 


2.0 


1.9 


2.0 


3.0 


2.0 


(recomputed)* 


18.0 


20.0 


20.0 


15.0 


20.0 


10.0 


9.5 


10.0 


45.0 


30.0 197.5 



•Recomputed cell score = (Performance weight X (adjusted cell score). In the event adjustments for data quality are not made 
the recomputed cell score = (pe rormance weight) X (normalized performance measure score). 



EXHIBIT 1 8— Continued 



Automotive 
Technician 
(normalized) 


1.0 


2.0 


2.0 


3.0 


2.0 


1.0 


2.0 


2.0 


1.0 


2.0 




(adjusted) 


0.8 


2.0 


2.0 


3.0 


2.0 


1 0 


1 0 


o n 
z.v 


a r\ 

I.U 


2.0 




(recomputed) 


8.0 


20.0 


20.0 


15.0 


40.0 


50 




IU.U 


\ O.U 


on r\ 

30.0 


172.5 


Computer 

Programmer 

(normalized) 


2.0 


3.0 


2.0 


3.0 


3.0 


1.0 


2.0 


3.0 


2.0 


2.0 




(adjusted) 


1.8 


3.0 


2.0 


3.0 


3.0 


1 0 


1 0 

I . Y 


o.u 


z.U 


2.0 




(recomputed) 


18.0 


30.0 


20.0 


15.0 


60.0 


50 




A ^ fi 
I o.u 


ou.u 


30.0 


232.5 


Word 
Processor 
(normalized) 


2.0 


2.0 


2.0 


1.0 


1.0 


2.0 


2.0 


1.0 


2.0 


1.0 




(adjusted) 


1.3 


20 


2.0 


1.0 


1.0 


20 


1 0 

I .7 


A fi 
I.U 


o n 
z.u 


1.0 




(recomputed) 


18.0 


20.0 


20.0 


5.0 


20.0 


100 


y.o 


o.u 


oU.U 


15.0 


152.5 


Medical Lab 
Technician 
(normalized) 


2.0 


2.0 


2.0 


2.0 


3.0 


20 


2.0 


2.0 


2.0 


2.0 




(adjusted) 


1.8 


2.0 


2.0 


2.0 


30 


90 


A O 






2.0 




(recomputed) 


18.0 


20.0 


20.0 


10.0 


60.0 


100 


y.o 


ac\ n 

IU.U 


oU.U 


or* n 
30. 0 


217.5 


Salesperson - 
retail 

(normalized) 


3.0 


2.0 


3.0 


2.0 


3.0 


2.0 


1.0 


2.0 


1.0 


10 




(adjusted) 


2.8 


2.0 


30 


20 


3.0 


2.0 


0.9 


2.0 


1.0 


1.0 




(recomputed) 


28.0 


20.0 


30.0 


10.0 


60.0 


10.0 


4.5 


10.0 


15.0 


15.0 


202.5 



EXHIBIT 18— Conf/nuec/ 



Electronic 
Technician 



(normalized) 


2.0 


3.0 


2.0 


3.0 


2.0 


2.0 


3.0 


2.0 


3.0 


3.0 


(adjusted) 


1.8 


3.0 


2.0 


3.0 


2.0 


2.0 


2.9 


2.0 


3.0 


3.0 


(recomputed) 


18.0 


30.0 


20.0 


15.0 


40.0 


10.0 


14.5 


10.0 


45.0 


45.0 247.5 
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Performance Indicators 


Performance Measures 


RANK 
ORDER 


SUM 


OCCUPATION 


1. Extent of rate of growth 

2. Extent of future demand 

3 Pyf Ant of fifflrm ntiv/A nr'tinn 

\J» 1 A IUI II Ul VJI | II 1 1 1 \J II Vw UvIM 1 

4. Extent to which staff are 
available 


a. Fastest growing 

a. Average annual openings 

a. evidence ot nondiscrimination 

a. Current and potential 
availability 


1 

2 
3 
4 
5 
6 
7 


247.5 
232.5 
217.5 
202.5 
197.5 
172.5 
152.5 


Electronic technician 
Computer programmer 
Medical lab technician 
Salesperson (retail) 
Machinist technician 
Automotive technician 
Wad processor 


5. Acceptability of costs 


a. Comparison of recurrent costs 
with other programs, locally 








6. Extent to which space is 
acceptable 


a. Comparison of space needs to 
other programs 








7. Extent to which enrollments 
likely to be sufficient 


a. Enrollment trends, statewide 








8. Extent to which training offered 
by other sources 


a. Other schools in LMA offering 
training for occupation 








9. Extent of placement in 
occupation for which trained 


a. Comparison of placement rates 
for similar programs, statewide 
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10. Extent of incentive for a. Number of inquiries about 

economic development training availability 

purpose 

O 

ERLC 



Between 1975 and 1985, the population in the SMSA expanded rapidly. It is estimated that the 
population in the SMSA grew at an average rate of 4.5 percent per year over the past decade as 
compared to an average rate of 4.1 percent for the state and 2.4 percentfor the nation asa whole. This 
growth in the population of the SMSA is attributed primarily to the fact that many out-of-state persons 
moved into the area. It is believed that this expansion in population will exceed an annual average of 
5 percent during the next several years. 

An increasingly larger proportion of the SMSA population falls into the 15- to 24-year-old age 
group, resulting in more pressure on the local educational agencies. At the same time, employers are 
finding that there are increasing numbers of persons available to enter the work force. 

The Grand SMSA work force grew at an average annual rate of 4.8 percent over the last 5 years. 
There is a noticeable migration of both workers and firms into the region. The creation of new jobs has 
kept pace with the increase in people seeking work, and as a result, unemployment is currently around 
6. 7 percent. This rate contrasts with the current state unemployment rate of 8.2 percent and the current 
national unemployment rate of 6.9 percent. Unemployment in the Grand SMSA has remained consis- 
tently below the state and national levels during most of the past decade. 

Labor is relatively inexpensive in the SMSA. For example, the average wage of production workers is 
$7.23 per hour. This figure compares to the national average wage of $8.61 per hour and to the state 
average wage of $7.47 per hour for production workers. Many firms have migrated to the area to take 
advantage of the supply of less expensive workers. Also, the pleasant climate and recreational 
opportunities have made it easy for firms to attract professional, managerial, and technical 
employees from across the nation. 

Many motels, restaurants, and retail stores that cater to an expanding tourist trade are located in 
the Grand SMSA, especially in Expansion City and its immediate surroundings. These businesses 
employ a major proportion of persons in the Grand SMSA work force. However, the levels of employ- 
ment in many of these businesses are subject to fluctuations because the tourist trade is seasonal in 
nature. These businesses, especially the restaurants and motels, use proportionately more unskilled 
labor than do other industries in the area. However, the presence of a sizable number of unskilled and 
undereducated persons in the area does provide a pool of available workers for businesses catering to 
the tourist trade. 

Employment in the manufacturing sector isexperiencing a steady growth. Aerospace, electronic, 
and glass firms employ the majority of employees in manufacturing. The growth of manufacturing in 
the SMSA is related to low energy costs, as well as to the availability of low-cost and reliable labor. 

Employers' Needs for Workers 



The Grand SMSA has 467 employers who employ three or more persons. These 467 employers 
provide 89 percent of the jobs in the SMSA The majority of employees are in the manufacturing, 
services, government, and wholesale and retail industries. The remainder are employed in transporta- 
tion, communications, construction, finance, insurance, real estate, and agriculture. 

The pattern of employment by industrial division in the Grand SMSA affects the distribution of 
occupational groups in this labor market area. Table 2 lists nine occupational groups and indicates 
the distribution of employment by groups for the SMSA and for the state. Concentration indices and 
estimates of growth and decline for each occupational grouping are also included. 
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TABLE 2 

PATTERNS OF EMPLOYMENT BY OCCUPATIONAL GROUP, GRAND SMSA 


Occupational 
Gtoud 


Percentage of Total Employment 
Local Statewide National 


Concentration Indexes 
Local/State Local/National 


Growth Predicted 
to 1990 


Prof a til on a \ 1 

rivivf f ivi iui/ 

Technical 
Workers 


12 


14 


16 


.86 


.75 


55% 


Managers, 

Officials, 

Proprietors 


13 


12 


11 


1.08 


1.18 


48% 


Sales Workers 


9 


9 


6 


1.00 


1.50 


32% 


Clerical 
Workerc 


20 


18 


18 


1.11 


1.11 


66% 


Crafts and 

Kindred 

Worker* 


11 


10 


13 


1.10 


.85 


28% 


Operatives 


12 


14 


15 


.86 


.80 


36% 


Service 
Workers 


18 


16 


14 


1.12 


1.29 


71% 


Laborers 
(nonfarm) 


4 


4 


4 


1.00 


1.00 


29% 


Farm Woncers 


1 


3 


3 


.33 


.33 


-8% 

(45% average of 
all occupations) 


NOTE Patterns of employment for occupations typically described in S f ate Employment Security Agency publications, which report occupational 
employment projections, both statewide and for selected labor market areas of a state. This information is sometimes found in the Annual Planning 
Report, which is also issued by this agency Concentration indexes can be calculated by the education agency. 


ERJC 
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Table 3 lists 19 specific occupations that require less than a community college degree as a 
prerequisite for employment and that are expected to have more than 100 job openings per year in 
the SMSA through 1990. 

TABLE 3 

AVERAGE ANNUAL OPENINGS 1965-1990 





Average Annual 




Average Annual 


Occupations 


Openings 


Occupations 


Openings 


Secretaries 


1608 


Carpenters/ Apprentices 


173 


Sales Workers 


1192 


Packers/Wrappers 


151 


Walters/Waitresses 


874 


Heavy Equipment 


138 


Cashiers 


647 


Mechanics 




Bookkeepers 


483 


Licensed Practical 


135 


Truck Drivers 


324 


Nurses 




Typists 


297 


Guards 


134 


Cooks 


261 


Delivery Route Workers 


123 


Assemblers 


234 


Auto Mechanics/ 


119 


Machine Operators 


198 


Apprentices 




Checkers/Examiners, 


180 


Receptionists 


107 



Mfg. 



NOTE Average annual job openings for many occupations are presented in State Employment Security Agency publications 
that report occupational employment projections, both statewide and for selected labor market areas of a state This kind of 
information may also be found in the Annual Planning Report that is issued by this agency 



Table 4 rank orders the 20 occupations in the SMSA that are expected to have the highest rates of 
growth for the 1985-1990 period. It should be noted that some of the occupations with high rates of 
growth have comparatively few average annual openings. This is understandable because relatively 
few persons are employed In these particular occupations. For example, although the rate of growth of 
opticians and lens grinders Is 1 1 7 percent, the average annual number of openings is only 14. This is 
because there are currently only 274 opticians and lens grinders in the SMSA 

People's Needs for Jobs 



There is little information available about what kinds of training are desired by the population of 
the SMSA. On the other hand, there is information that suggests that certain groups in the community 
have not shared in its economic growth. For example about 13 percent of all persons in Grand are 
considered to be living at or below the poverty level. Approximately two-thirds of these persons are 
nonwhites. The active file of the local state employment service office indicates that of those persons in 
this group who are seeking employment, more than 80 percent are on welfare. In addition, few 
applicants from this group of persons seem to find continuous substantial gainful employment even if 
they are referred to an employer. The facts indicate that the poorin the area are much more likely to be 
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TABLE 4 

TOP 20 OCCUPATIONS— RATE OF GROWTH 



Estimated Rate 
of Growth 1 985-90 Average Annual 
Occupations < a$ a percentage) Openings 



Dental Hygienists 

Data Processing Machine Repairers 
Clerical Assistants 
Child Care Workers 
LPNs 

Health Record Technicians 
Teacher Aides 

Boarding/Lodging Housekeepers 

Job, Die Setters. Metal 

Animal Caretakers 

Dental Assistants 

Opticians/Lens Grinders 

Receptionists 

Nurses Aides/Orderlies 

Legal Secretaries 

Lodging Quarters Cleaners 

Chemical Technicians 

Billing Clerks 

Ushers: Recreation/Amusement 



300 


47 


236 


32 


200 


12 


186 


84 


151 


135 


150 


• 


145 


96 


133 


22 


125 


13 


123 


54 


120 


75 


117 


14 


112 


107 


109 


84 


104 


68 


103 


94 


100 


• 


100 


52 


100 


17 



NOTE' Rates of growth in the number of job openings for a number of occupations are reported in State Employment Security 
Agency publications that report occupational employment projections, both statewide and for selected labor market areas of 
a state 



"Less than 10 openings per year 



chronically unemployed, and they are in need of job skills that will qualify them, or increase their ability 
to compete, for stable employment. 

The cost of living has been increasing very rapidly in the SMSA even though inflation is under better 
control nationally. As a result, many families are required to have more than one wage earner 
Although reliable statistics are difficult to obtain, the local state employment services office estimates 
that about 200 to 250 full-time homemakers will be entering the work force during the next year. That 
office also estimates that most of these persons will be without significant job skills. 

During the past 1 2 months, several employers serving on Community College vocational educa- 
tion advisory and craft committees have indicated an interest in upgrading skills of some of their 
present employees. However, the specific kinds of training desired by these and other employers and 
the number of employees who might be interested in upgrading their skills are not fully known at this 
time. 
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Capacity to Meet Needs for Training 



At the secondary level there are a total of nine comprehensive high schools in the three counties. 
At the postsecondary level, there is Grand Community College. Vocational education programs are 
offered at night to adults by Expansion City public schools. 

Compared to the state averages, the percentage of persons who are enrolled in vocational 
programs in grades 9 through 12 and in the community college is quite low. Two facts may be 
contributing to this state of affairs. First, the relative affluence of the SMSA has historically been 
associated with a large percentage of high school students making plans for college rather than for 
employment directly following graduation from school. Second, the rapid expansion of the 
school-age population has put a continuing construction and financial burden on the school 
systems in all three counties, especially in Grand County. 

Training-related placement rates for technology programs are relatively high at the community 
college. About 78 percent of those completing training and seeking employment find training-related 
employmentwithin three months after graduation. Tb^ lowest rate is62 percentforthe chemical technol- 
ogy program, whereas the highest rate is 94 percenf for the electronic technology program. The 
training-related placement rate for all program completers at the college (excluding those in courses 
to upgrade existing skills) is approximately 55 percent. 

The various local advisory committees for vocational education in the SMSA have indicated that 
vocational education programming should be expanded. Currently there are long lists of people 
waiting to enroll in quantity food production programs at the high schools and in the hospitality 
management course at Grand Community college. However, a lack of money is an important 
constraint in decisions that have been made by local school boards and the community college 
about expanding existing vocational education programs or starting new ones. The rapid growth in 
the elementary school-age population over the last few years has caused most of the school systems in 
the SMSA to give their highest priority for funding to the building and remodeling of elementary school 
buildings, whereas the community college's highest funding priority has been to upgrade existing 
vocational education equipment to keep pace with changing technolog ies in the workplace. The tax 
base in the three counties has expanded but the rising population growth and inflation leave most of 
the school systems in the SMSA with continuing financial difficulties. 
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Examples of Information Categories, Performance 
Indicators, Performance Measures, 
and Performance Scores 



Note: The parentheses are used to denote options within a given performance measure 
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Information and Data Selection Framework 



Purpose 



Planning 
Components 



Employment 
Context 



A 
N 

N 

i 

N 

G 



Information 
Categories 



Job Opportunities 



Performance Indicators 



Whether current demand for new workers 
in this occupation is sufficient fa imple- 
menting a program 



Whether rate of growth is sufficient to war- 
rant implementing a program 



Whether future demand for workers in this 
occupation is great enough fa implement- 
ing a program 
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Information and Data Selection Framework 



Performance Measures and Scores 
(Using the Pass/Fail Normalization Procedure) 



1 if the local advisory council for vocational education (oratrade association) indicatesthat there is, 
and there is likely to continue to be, a shortage of trained persons available for employment in this 
occupation in this labor market area (or several major labor market areas of the state) during the 
next 12 months (or some other time frame) 

0 if the local advisory councilfor vocational education (or a trade association) indicates that there is 
not likely to be a shortage of trained persons available for employment in this occupation in this 
labor market area (or in several major labor market areas of the state) during the next 1 2 months 
(or some other time frame) 



1 if the reported average annual number of job openings in this occupation is greater than the 
numberofcompletersinthe past 12 months (or some other time frame) from all training programs 
in this labor market area (or statewide) that prepare persons for this occupation 

0 if the reported average annual number of job openings in this occupation is less than or equal to 
the number of completers in the past 12 months (or some other time frame) from all training 
programs in this labor market area (or statewide) that prepare persons for this occupation 



1 if this occupation is among the 1 0 (or some other number) fastest growing occupations in this labor 

market area (or the state or nation if local data are unavailable) 
0 if this occupation is not among the 10 (or some other number) fastest growing occupations in this 

labor market area (or the state or nation if local labor market area data are unavailable) 



1 if the estimated number of job openings in this occupation during the next 12 months (or some 
other timeframe) in this labor market area is greater than the estimated number of graduates from 
programs that prepare persons for this occupation in this labor market area 

0 if the estimated number of job openings in this occupation during the next 12 months (or some 
other time frame) in this labor market area is equal to or less than the estimated number of 
graduates from programs that prepare persons for this occupation in this labor market area 



1 if growth prospects, nationally, for this occupation are considered above average 

0 if growth prospects, nationally, for this occupation are considered average or below average 



1 if trade or business-industry associations, unions, the local vocational education advisory council, a 
program craft committee, or other sources indicate that there will likely be an increased demand 
for new workers in this occupation in this labor market area (or else statewide or nationally) 
because of the adoption of newtechnologies by firms (e.g., technicians to repair industrial robots) 

0 if trade or business-industry associations, unions, the local vocational education advisory council, a 
craft committee, or other sources indicate that there will likely not be an increased demand for new 
workers in this occupation in this labor market area (or else statewide or nationally) because of the 
adoption of new technologies by firms (e.g., technicians to repair industrial robots) 



1 if this occupation is new to this labor market area and the estimated number of new jobs in this 

occupation will be sufficient to justify implementing a training progrcm 
0 if this occupation is new to this labor market area but the estimated number of new jobs in this 

occupation will not be sufficient to justify implementing a training program 
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Information and Data Selection Framework 





Conditions of 
Employment 


Whether the principles of affirmative action 
in hiring persons for this occupation are 
practiced by employers in this labor market 
area 


Whether employment in this occupation is 
generally stable 


Location of 
Employment 


Whether jobs for program completers are 
likely to be conveniently located 


Vocational 

Education 

Processes 


Staffing 


Whether staff are readily available to teach 
the skills required by this occupation 


Whether any new staff that are needed to 
provide instruction in the skills required by 
this occupation can be easily recruited 
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Information and Data Selection Framework 



1 If it Is known that most firms (and/or the largest firms) in this labor market area do not discriminate 
against persons from minority groups when hiring for jobs in this occupation 

0 if it is known that mostfirms (and/or the largest firms) in this labor market area discriminate against 
persons from minority groups when hiring for jobs in this occupation, or there is insufficient evidence 
upon which to make a Judgement about discrimination in hiring persons from minority groups 



1 if it is known that mostfirms (and/or the largest firms) in this labor market area do not discriminate 
against physically (or mentally) handicapped persons when hiring for jobs in this occupation 

0 if it isknown that mostfirms (and/or the largest firms) in this labor market area discriminate against 
physically (or mentally) handicapped persons when hiring for jobs in this occupation, or there is 
insufficient evidence upon which to make a judgement about discrimination in hiring physically 
(or mentally) handicapped persons. 



1 if it is known that mostfirms (and/or the largest firms) in this labor market area do not discriminate 
against persons with English language difficulties when hiring for jobs in this occupation 

0 if it is known that mostfirms (and/or the largest firms) in this labor market area discriminate against 
pesons with English language difficulties when hiring for jobs in this occupation, or there is insuffi- 
cient evidence upon which to make a judgement about discrimination in hiring persons with 
English language difficulties 



1 if persons employed in this occupation in this labor market area (or statewide) typically enjoy 

continuing employment (i.e., layoffs In this occupation are usually very infrequent and of short 

duration or else virtually nonexistent) 
0 if persons employed in this occupation in this labor market (or statewide) typically ex 

or two layoffs each year; or the frequency of layoffs is unpredictable, but when layoffs occur, they 

are likely to be prolonged 



1 if, in this labor market area (or statewide), this occupation is regarded as recession proof (i.e., 
employment in this occupation is usually unaffected by even the mildest recessions or downturns in 
the economy) 

0 if, in this labor market area (or statewide), this occupation is regarded as partially recession proof 
(i.e., employment in the occupation is sometimes but not always affected by mild recessions or 
downturns in the economy) or vulnerable to recession (i.e., employment in this occupation is 
typically adversely affected by even mild recessions or downturns in the economy) 

1 if most (or the largest) employers who hire persons for this occupation in this labor market area are 
located near public transportation 

0 if most (or the largest) employers who hire persons for this occupation in this labor market area are 
not located near public transportation 



1 if there are school staff available who can be assigned to a program to train persons for this 
occupation 

0 if thee are either too few or no school staff who can be assigned to a program to train persons for this 
occupation 



1 if there is reason to suppose that newteaching staff can be recruited for a program to train persons 
for this occupation 

0 if there Isreason tosupposethat recruiting newteachingstafffora program to train persons forthis 
occupation will be difficult, or the ability to successfully recruit is uncertain 
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Information and Data Selection Framework 





Whether staff who will be used to train stu- 
dents for this occupation are able to teach 
effectively without additional training 


Facilities 


Whether a facility to train persons for this 
occupation presently exists or can be easily 
obtained 


Whether the space required to train per- 
sons for this occupation Is acceptable 


Dollars 


Whether the estimated start-up cost for a 
program to train persons for this occupa- 
tion is acceptable 


Whether the estimated recurrent cost for a 
program to train persons for this occupa- 
tion is acceptable 
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1 if staff to be assigned will likely require additional instruction to be able to teach effectively 
0 if staff to be assigned will not require additional instruction to be able to teach effectively 



1 if school-owned facilities could be used to train students for this occupation 
0 if school-owned facilities could not be used to train students for this occupation 



1 if equipment required to train persons for this occupation could be procured within the next 3 
months (or some other number of months) and the cost of such equipment does not exceed the 
mean (or median) cost of equipment for all other training programs in this school (or would not 
exceed the cost of equipment required by any of the other candidate occupations) 

0 if equipment required to train persons for this occupation could not be procured within the next 3 
months (or some other number of months); or, even if the equipment could be procured, the cost of 
such equipment would exceed the mean (or median) cost of equipment for all other training 
programs in this school (or would exceed the cost of equipment required by any of the other 
candidate occupations or the mean or median for all of the other candidate programs) 



1 if less space would be required for a training program for this occupation than the mean (or 

median) space used for occupational training programs in this school 
0 if the spaces required for a training program for this occupation would be equal to or exceed the 

mean (or median) space used for occupational training programs in this school 



1 if the space required for a training program for this occupation would be less than the space 
required by any of the other candidate occupations; or, the space required would be less than the 
mean (or median) space required by all of the other candidate occupations 

0 if the space required for a training program for this occupation would be equal to or more than the 
space required by any of the other candidate occupations, or the space required would be equal 
to or more than the mean (or median ) space required by all of the other candidate occupations) 



1 if the estimated start-up cost (other than for facilities) for a typical program to train students for this 
occupation in this state is less than the mean (or median) start-up cost for occupational programs 
in this school 

0 if the estimated start-upcost (otherthanfor facilities) fora typical program totrain students for this 
occupation in this state is equal to or greater than the estimated mean (or median) start-up cost 
for programs associated with all of the other candidate occupations 



1 if the estimated recurrent cost (other than for facilities) for a typical program totrain students in this 
occupation in this state is less than the mean (or median) recurrent cost for occupational 
programs in this school 

0 if the estimated recurrent cost (other than for facilities) fora typical program totrain students in this 
occupation in this state is equal to or greater than the mean (or median) recurrent cost for 
occupational programs in this school 



1 if theestimated recurrent cost (otherthanforfacilities)fora typical program totrain students for this 
occupation in this state is less than the estimated mean (or median) recurrent cost for training 
programs associated with all of the other candidate occupations 

0 if the estimated recurrent costs (other than for facilities) for a typical program to train students for 
this occupation in this state is equal to or greater than the estimated mean (or median) recurrent 
costs for training programs associated with all of the other candidate occupations 
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Information and Data Selection Framework 





Curriculum 


Whether curriculum is available or must be 
developed 




Enrollment 
Conditions 


Whether facilities to train persons for this 
occupation are designed to accommo- 
date physically handicapped students 




Student Interest 


Whether enrollment can be expected to 
be sufficient to justify implementing a pro- 
gram to train persons for this occupation 


Vocational 
Education Outputs 


Availability of 

Training 

Opportunities 


Whether there are other institutions provid- 
ing training for this occupation 




Potential Outputs 


Whether the completion rate for a program 
to prepare persons for this occupation is 
likely to be favorable 


Vocational 
Education 


Potential for 
Placement 


Whether employers have been identified 
who will want to hire the completers of a 
program that prepares persons for this 
occupation 
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Information and Data Selection Framework 



1 if a curriculum to train students for this occupation is readily available 

0 if a curriculum to train students for this occupation is not readily available and curriculum devel- 
opment activities will be required 



1 if the facility that will likely be available for a program to train persons for this occupation is 

designed to be accessible to persons with physical handicaps 
0 if the facility to be used is not designed for access by the physically handicapped 



1 if the equipment commonly used in this occupation can be operated by physically handicapped 
persons as required to meet production quotas and without special modifications to the 
equipment 

0 if the equipment used in this occupation can be operated by physically handicapped persons, 
but only with special modifications 



1 if there have been sufficient requests from the general public to warrant a program for this 
occupation 

0 if there have been either too few or no requests from the general public to warrant a program 



1 if there has been a trend toward increased enrollments in programs to train persons for this 

occupation over the past few years, statewide 
0 if enrollments to train persons for this occupation have remained stable, are decreasing, or tend to 

be unpredictable, statewide 



1 if there are no other education agencies or institutions offering training for this occupation in this 
labor market area 

0 if there are other agencies or institutions offering such training in this labor market area 



1 if the completion rate, statewide, for programs at this educational level that train persons for this 
occupation is greater than the completion rate for the program's field (e.g., the completion rate for 
all vocational education programs in the health occupations field at this education level) 

0 if the completion rate, statewide, for programs at this education level that train persons for this 
occupation is equal toor less than the completion rate for the program's field (e.g., the completion 
rate for all LPN programs at this education level, statewide, is equal to or less than the completion 
rate for all vocational education programs in the health occupations field at this education level) 



1 if the completion rate, statewide, for programs at this education level that train persons for this 

occupation is greater than the completion rate, statewide, for programs at this education level 

that train persons for any of the other candidate occupations 
0 if the completion rate, statewide, for programs at this education level that train persons for this 

occupation is equal toor less than the completion rate, statewide, for programs at this education 

level that train persons for any of the other candidate occupations 



1 if the school has received requests from firms in the past 2 months (or some other number of 
months) to train persons for or upgrade the existing skills of persons in this occupation 

0 if the school has not received requests from firms in the past 2 months (or some other number of 
months) to train persons for or upgrade the existing skills of persons in this occupation 



Information and Data Selection Framework 



Benefits 


Economic 


Whether the (availability of persons trained 






for this occupation is likely to either interest 






firms to relocate into the local labor market 






area or encourage locaiiy based firms to 






remain or expand their operations 
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Information and Data Selection Framework 



1 if the state (or else a local or regional) economic development agency has received requests from 
one or more local employers or from employers who have expressed an interest in locating here 
about the availability of, or possibility of implementing, a vocational education program to train 
persons for this occupation 

0 if the state (or else a local or regional) economic development agency has not received requests 
from one or more local employers or from employers who have expressed an interest in locating 
here about the availability of, or possibility of implementing, a vocational education program to 
train persons for this occupation 



Information and Data Selection Framework 



Purpose 


Evaluation 
Components 


information 
Categories 


Performance indicators 


E 

9m 

V 


Context 




Tho ovtont to VA/hich ouiTAnt fiftmnnn! for 

1 1 Is? DaIUI II IU Wl II wl 1 wUllvvl II Uvl 1 t\Jl IU • wi 

new workers in this occupation is greater 
than the supply 


A 








L 








U 








A 
T 
1 

o 






The extent to which the rate of growth in the 
occupation for which students are being 
trained warrants continuing this program 








N 




• 










The extent to which future demand for 
workers in this occupation is great enough 
to consider continuing this training 
program 
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Information and Data Selection Framework 



Performance Measures and Scores 
Using the Range Normalization Procedure) 



3 if the reported average annual number of job openings in the occupation for which students in this 
program are being trained is greater than the number of completers from all programs of this kind 
in this labor market area (or statewide) in the past 12 months (or some other time frame) 

2 if the reported average annual number of job openings in the occupation for which students in this 
program a r e being trained is equal to or ve;/ nearly equal to the number of completers from all 
programs of this kind in this labor market area (or statewide) in the past 12 months (or some other 
time frame) 

1 if the reported average annual number of job openings in the occupation for which students in this 
program are being trained is less than the number of completers from all programs of this kind in this 
labor market area (or statewide) in the past 12 months (or some other time frame) 



3 if this training program prepares persons for an occupation that is among the 1 0 (or some other 

number) fastest growing occupations in this labor market area (or the state or nation if local labor 

market area data are unavailable) 
2 if this training program prepares personsfor an occupation that is noi among the 1 0 (or some other 

number) fastest growing occupations in this labor market area (or the state or nation) but its rate of 

growth Is projected as increasing 
1 if this training program prepares persons for an occupation whose projected rate of growth is 

expected to be either constant or decline 



3 if growth prospects, nationally, for the occupation for which students in this program are being 
trained are considered above average 

2 if growth prospects, nationally, for the occupation for which students in this program are being 
trained are considered average 

3 if growth prospects, nationally, for the occupation for which students in this program are being 
trained are considered below average 



3 if the local vocational education advisory council (or a program craft commirtee, or trade 
association) indicates that there is likely to be a continuing shortage of trained persons available 
for employment in this labor market area (or in several labor market areas of the state) during the 
next 12 months (or some other time frame) in the occupation for which this program prepares 
students 

2 if the local vocational education advisory council (or program craft committee or a trade associa- 
tion) indicates that there will likely be periodic or spot shortages of trained persons available for 
employment in this labor market area (or in several labor market areas of the state) during the next 
12 months (or some other time frame) in the occupation for which this program prepares students 

1 if the local vocational education advisory council (or program craft committee or trade associa- 
tion) indicates that there is not likely to be a shortage of trained persons available foremployment 
in this labor market area (or in several labor market areas of the state) during the next 12 months (or 
some other time frame) in the occupation for which this program prepares students 
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Information and Data Selection Framework 



The extent to which employers' requests for 
trained workers in this occupation warrants 
continuing this training program 



The extent to which the principles of affirm- 
ative action in the hiring of persons for this 
occupation are practiced by employers 



The extent to which employment ; s stable 
in the occupation for which This program 
trains students 
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Information and Data Selection Framework 



3 if more employers requested completers from this progrom in the past 12 months (or some other 
time frame) than from any other program being evaluated (or from any other occupational 
program in this school) 

2 if the number of employers requesting completers from this program in the past 12 months (or some 

other timeframe) exceeded the mean (or median) of the other programs be.ng evaluated (or for 

all other occupational programs in this schooi) 
1 if the number of employers requesting completers from this program in the past 12 months (or some 

othei time frame) was equal to or less than the mean (or median) for the other training programs 

being evaluated (or for all other occupational programs in this school) 



3 if it is known that most firms (and/or the largest firms) in this labor market area hiring persons for the 
occupation for which students in this program are being prepared do not discriminate against 
minority group members 

2 if it is known or strong Iv suspeci ed that most firms (and/or the largest firms ) in this labor market area 
hiring persons for the occupation for which students in this program are being prepared are 
reluctant to hire persons from minori ty groups, or else tnere is insufficient information upon which to 
make a judgement about discrimination in hiring minority group members 

1 if it is known that most th ns (and/or the largest firms) in this labor market area hiring persons for the 
occupation for which students in this program are being prepared discriminate against minority 
group membeis 



3 if it is known that most firms ( and/or the largest firms) in this labor market area hiring persons for the 
occupation for which students in this program are being prepared do not discriminate against 
persons with limited English language proficiency 

2 if it is known or strongly suspected that most firms (and/or the largestfirms) in this labor market area 
hiring persons foi the occupation for which students in this program are being prepared are 
reluctant to hire persons from minorty groups, or else there is insufficient information upon which to 
make a judgement about discrimination in hiring persons with limited English language 
proficiency 

1 if it Is krown that most firms (and/ or the largestfirms) in this labor market area hiring persons for the 
occupation for which 3*udenft in this program are being prepared discriminate against persons 
with limited English language proficiency 



3 it persons employed if i tnL labor market area in the occupation for which students in this program 

are being prepared Typically an,c/ continuing employment (i.e., layoffs in this occupation are 

usually very infrequent one! of short duration or are virtually nonexistent) 
2 if persons employed in this labor r.a^et aroa in the occupation for which students in this program 

cue be ! ng prepared typicaiiy exoenence one a two layoffs each but unemployment rarely lasts 

rnofe than a few w3eks 

1 if persons emoloyed in this !abo» market in + he occupation for which students in this program are 
toeing prepared can expect layoffs that are typically frequent and/or prolonged 



3 if. In iftjs iaoor marker area (or stato\vd\*;, fne occupation for which this program trains persons is 

'eijcii'ddd :6cessicnproof (i.e employment :.-> the occupation is usually unaffected by even the 

mildest recessions or downturns in the economy) 
2 if, in this labor market a r ea (or statewide), the occupation for which this program trains persons is 

regarded as partially recessionproof (!.a. employment in the occupation is sometimes but not 

always affected by miid recessions or downturns in the economy) 
1 if. in this Jabor market area (o; s+otowtde), the occupation for wftic. , fhis program trains persons is 

regarded as vulnerable to recession (i.e. employment in the occupaiion is typically adversely 

affected by even mild recessions cr downturns in . f he economy*) 
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Information and Data Selection Framework 





Location of 
Employment 


The extent to which program completers 
are likely to find that job openings are con- 
veniently located 


Vocational 

Education 

Processes 


Expenditures 


The extent of annual recurrent costs for this 
program 




Staffing 


The extent to which staffing to operate this 
program is adequate 




Facilities 


The extent to which facilities to operate this 
program are adequate 




Curricula 


The extent to which this program's curricu- 
lum is adequate 
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Information and Data Selection Framework 



3 if more than 2/3 (or some other fraction denoting a majority) of this program's completers reported 

(in the most recent follow-up study) that when they went looking for a job they found that most 

prospective employers could be reached by public transportation 
2 if anywhere from 1/3 to 2/3 of this program's completers reported (n the most reent follow-up study) 

that when they went looking for a job they found most prospective employers could be reached by 

public transportation 

1 if less than 1/3 of this program's completers reported (in the most recent follow-up study) that when 
they went looking for a job they found that most prospective employers could be reached by 
public transportation 



3 if the cost of operating this program (on some comparative basis, ag., full-time equivalent (FTE), 
per-student or program basis) is less than that of any of the other programs being evaluated (or 
any other occupations program in this school) 

2 if the cost of operating this program (on some comparative basis, e.g.. FTE, per-student, or program 
basis) is less than the average (e.g., mean, median) comparable cost of all the other training 
programs being evaluated (or less than the average for all of the other occupational programs in 
this school) 

3 if the cost of operating this program (on some comparative basis, e.g., FTE, per-student, or program 
basis) is more than the average (e.g., mean, median) comparable cost of all the other training 
programs being evaluated (or less than the average for all of the other occupational programs in 
this school) 



3 if in the judgement of the vocational director or other person responsible for staff supervision there 
continues to be a sufficient number of qualified staff to operate this program 

2 if in the judgement of the vocational director or other person responsible for staff supervision there 
continues to be a sufficient number of qualified staff to operate this program but not all of the staff 
are able to do an effective job 

1 if maintaining enough staff tooperate this program has been a continuing or intermittent problem 



3 if the equipment used for this program is judged by the program's craft committee as being more 

than minimally adequate for preparing students for employment 
2 if the equipment used for this program is judged by the program's craft committee as being 

minimally adequate for providing students with the kinds of job skills they will require for 

employment 

1 if the equipment used for this program is judged by the program's craft committee as being less 
than minimally adequate for preparing students for employment 



3 if this program's curriculum is judged by its craft committee as being more than adequate for 

providirg students with the kinds of job skills they will require for employment 
2 if this program's curriculum is judged by its craft committee as being minimally adequate for 

providing students with the kinds of job skills they will require for employment 
1 if this program's curriculum is judged by its craft committee as being less than adequate or 

possessing serious deficiencies in specific areas for providing students with the kinds of job skills they 

will require for employment 
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| Enrollment 
Conditions 


The exient to which this program has 
accommodated physically handicapped 
students 






The extent to which this program has 
accommodated persons with limited 
English-language proficiency 






The extent to which this program has 
accommodated educationally disadvan- 
taged students 




^tllHftnt Intoroct 

JtUUolll HlfOIOOl 


The extent of enrollment in this program 


Vocational 

Education 

Output 


Trained Supply 


1 he extent to which this program is success- 
ful in turning out completers 
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3 if there is tangible evidence that this program has made provisions for including physically (or 
mentally) handicapped students in training classes attended by nonphysically (or nonmentally) 
handicapped students 

2 if there is tangible evidence that this program has made provisions for training physically (or 
mentally) handicapped students but not in training classes attended by nonphysically (or men- 
tally) handicapped students 

1 if there is no tangible evidence that this program has made provisions for training physically (or 
mentally) handicapped students 



3 if there is tangible evidence that this program has made provisions for teaching students with 
limited English language proficiency and including these students in regular classes 

2 if there is tangible evidence that this program has made provisions for teaching students with 
limited English language proficiency but not including these students in regular classes 

1 if there is no tangible evidence that this program has made provisions for teaching students with 
limited English language proficiency 



3 !f there is tangible evidence that this program has made provisions for including educationally 
disadvantaged students in classes attended by non-educationally disadvantaged students 

2 if there is tangible evidence that this program has made provisions for educationally disadvan- 
taged students but only in special classes 

1 if there is no tangible evidence that this program has made provisions for training educationally 
disadvantaged students 



3 if the number of persons requesting enrollment in this program (as their first choice) during the past 

semester(oryearor some other time frame) exceeded that of any other program being evaluated 

(or any other occupational program in this school) 
2 if the number of persons requesting enrollment in this program (as their first choice) exceeds the 

mean (or median) enrollment for all of the other programs being evaluated (or for all other 

occupational programs in this school) 
1 if the number of persons requesting enrollment in this program (as their first choice) is less than the 

mean (or median) enrollment for all of the other programs being evaluated (or for all other 

occupational programs in this school) 



3 if the completion rate of this program is greater than the completion rate of any other program 
being evaluated (or for all other occupational programs in this school or for this type of program at 
this education level, statewide; or for all vocational education programs in the same program field 
at this education level statewide; or for all vocational education programs at this educational 
level statewide) 

2 if the completion rate for this program exceeds the mean (or median ) completion rate of any other 
program being evaluated (or for all other occupational programs in this school; or for this type of 
program at this education level statewide; or for all vocational education programs in the same 
program field at this education level statewide; or for all vocational education programs at this 
education level statewide) 

1 if the completion rate for this program is less than the mean (or median) completion rate for other 
programs being evaluated (or for all other occupational programs in this school; or for this type of 
program at this educational level statewide; or for all programs in the same program field at this 
education level statewide) 
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Vocational 

Education 

Outcome 


Placement 
Experiences 


The extent to which this program's complet- 
ers are successful in finding employment 


The extent of time that it takes this pro- 
gram's completers to find employment 
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Information and Data Selection Framework 



3 if the completer-toenrollee ratio in this program in the last semester (or school year or some other 
time frame) exceeds the ratio of any other program being evaluated (or for all occupational 
programs in this school) 

2 if the completer-to-enrollee ratio in this program in the last semester (or school year or some other 
time frame) exceeds the mean (or median ) ratiofor all of the other programs being evaluated (or 
for all occupational programs in this school; or for this type of program, statewide; or for all 
vocational education programs in the same program field at this education level, statewide; or for 
all vocational education programs at this education level, statewide) 

1 if the completer-to-enrollee ratio in this program in the last semester (or school year or some other 
time frame) is less than the mean (or median) ratiofor all of the other programs being evaluated 
(or for all occupational programs in this school; or for this type of program, statewide; or for all 
vocational education programs in the same program field at this education level, statewide; or for 
all vocational education programs at this education level, statewide) 

3 if the percentage of persons who completed this program in the last semester (or school year or 
some other time fram j) who found jobs in the occupation for which they were trained (or a 
training-related occupation) exceeds the percentage of any other program being evaluated (or 
for all occupational programs in this school; or for all vocational education programs in the same 
program field at this educational level, statewide; or for all vocational education programs at this 
education level statewide) 

2 if the percentage of persons who completed this program in the last semester (or school year or 
some other time frame) who found jobs in the occupation for which they were trained (or a 
training-related occupation) exceeds the mean (or median) percentage for all of the other 
programs being evaluated (or for all occupational programs in this school; or for this type of 
program, statewide; or for all vocational education programs in the same program field at this 
education level, statewide; or for all vocational education programs at this education level, 
statewide) 

1 if the percentage of persons who completed this program in the last semester (or school year or 
some other time frame) who found jobs in the occupation for which they were trained (or in a 
training-related occupation) is less than the mean (or median) percentage for all other programs 
being evaluated (or for all occupational programs in this school; or for this type of program at this 
education level, statewide; or for all vocational education programs in the same program field at 
this education level, statewide; or for all vocational education programs at this education level, 
statewide) 



eric 



if the mean job search time for persons who completed this program last semester (or school year or 
some other time frame) and who sought employment was less than that for any other program 
being evaluated (or of any other occupational program in this school for which such data are 
available; or is at least 1 standard deviation above the mean for all of the occupational progrcms 
in this school for which such data are available 

if the mean job search time for persons who completed this program last semester (or school year or 
some other time frame) and who sought employment was less than the mean (or median) job 
search time for all of the other programs being evaluated (or exceeded the mean or median job 
search time of all of the occupational programs in v his school for which data are available; or is 
between +1 and -1 standard deviation from the mean search time for all of the occupational 
programs in this school for which such data are available) 

if the mean job search time for persons who completed this program last semester (or school year or 
some other time frame) and who sought employment exceeded the mean (or median) job search 
time for all of the other programs being evaluated (or exceeded the mean or median job search 
time of all of the occupational programs in this school for which data are available; or is more than 
1 standard deviations below the mean search time for all of the occupational programs in this 
school for which such data are available) 
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Information and Data Selection Framework 



Benefits from 

Vocational 

Education 


Economic/Societal 
Benefits 


The extent to which the availability of this 
program has been an inducement for 
bringing new firms into the local labor 
market area or encouraging existing firms 
to remain or expand their operations 






The extent to which this program has been 
responsible for reducing dependence on 
public assistance 




Personal Benefits 


The extent to which program completers 
benefit in terms of salaries or wages 
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3 if a local- (regional- or state-) level economic development agency has requested this school to 
make this program available to one or more firms in order to induce them to locate in this labor 
market area, encourage them to remain in this area, or expand their operations and the program 
was made available 

2 if a local- (regional- or state-) level economic development agency has requested this school to 
make this program available to one or more firms in order to induce them to locate in this labor 
market area, encourage them to remain in this area, or expand their operations but the program 
could not be made available to them in a timely way 

1 if a local- (regional- or state-) level economic development agency has requasted this school to 
make this program available to one or more firms in order to induce them to locate in this labor 
market area, encourage them to remain in this area, or expand their operations b'Jr the request 
was rejected 



3 if this program trains dislocated workers or chronically unemployed or underemployed persons and 
the results of the most recent follow-up of program completers indicate that the percentage of 
completers no longer requiring public assistance exceeds the percentage of any similar programs 
in this labor market area 

2 if this program trains dislocated workers or chronically unemployed or underemployed persons and 
the results of the most recent follow-up of program completers indicate that the percentage of 
completers no longer requiring public assistance exceeds the mean (or median) percentage for 
all similar programs in this labor market area 

1 if this program trains dislocated workers or chronically unemployed or underemployed persons and 
the results of the most recent follow-up of program completers indicate that the percentage of 
completers no longer requiring public assistance equals or is less than the mean (or median) 
percentage for similar programs in this labor market area 



3 if the results of the most recent follow-up study of program completers indicate that completers 
from this program finding employment in an occupatio, for which they were trained (or in 
training-related employment or both) had mean entry-level (or seniority or reentry) earnings that 
exceeded the mean (or seniority or reentry) earnings of any of the other programs being evaluated 

2 if the results of the most recent follow-up study of program completers indicate that completers 
from this program finding employment in the occupation for which they were trained (or in 
training-related employment or both) had mean entry-level (or seniority or reentry) earnings that 
exceeded the mean entry-level (or seniority or reentry) earnings of all of the other programs being 
evaluated 

1 if the results of the most recent follow-up study of program completers indicate that completers 
from this program finding employment in the occupation for which they were trained (or in 
training-related employment or both) had mean entry-level (seniority or reentry) earnings that 
were less than the mean entry-level (or seniority or reentry) earnings of all of the other programs 
being evaluated 
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AppendiH B 

Examples of Databases and Sources 



Appendix B helps users locat9 sources of information to use with the information and data 
selection framework and to develop a context description, It focuses on four types of information: 
( 1 ) the context for employment, (2) the needs of employers for trained workers, (3) the needs of people 
for job skills, and (4) the capacity of the education system to meet the job skill needs of employers and 
individuals, 

The source listing format resembles a "laundry list/' At first glance, there may seem to be substan- 
tial duplication among the listings, However, this approach provides a broad array of information and 
agency alternatives and should be especially useful for individuals at the local level who may be 
unfamiliar with sources of employment and education information and the agencies that collect and 
publish it, 

Not all of the information or sources listed will be available in all geographic areas of the country, 
For a major urban or standard metropolitan statistical area, most, or all, of the listed information is 
available. In contrast, for the more rural, sparsely populated areas of the country, little of the informa- 
tion may be available, For this reason, as many alternative sources as possible have been included, 
Whan information is not available from one agency, some resourceful "detective" work may uncover it 
at another. In the end, individual administrators, planners, or evaluators are responsible for choosing 
the information or data. These persons will have to rely on the very best sources available in their local 
areas. 



Source* This appendix is a modification of one found in Starr, Merz, and Zahniser (1982) 
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Defining the Context for Employment 

Information Element Questions to Be Answered Information Purpose 

Population Characteristics what is the trend of population and To indicate the approximate size of 

employment growth in the commu- the total labor force 
nity or region served by the education 
agency? 



What has been the trend in the age To show the potential market for the 
groups served by this institution? services of particular institutions 



How do the population trends com- 
pare with the averages for the state 
and the nation? 



To give an indication of the degree to 
which in- or out-migration of popula- 
tion is affecting the community 
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Defining the Context for Employment 



Information Sources 


Agencies Supplying Information 


rAIlMUUI {JlUMMiMy lt?pOfT 


State employment security agency 


Current population report 


U.S. Department of Commerce, Bureau af Census 




State departments af health and planning or community 


State departments of health (bureau of vital statistics) and 


development 


olnnninfl Of f^fVY"! mi i n i fv /Hovyoloopn/snf nnonrioc— H.itn 
\ji\jt ii in tyj i ii i iui ii i y • mi u uyt7ML>it79 U\JIQ 




and/or studies 




Regional and local planning commissions—annual reports 


Regional and local planning commissions 


Economically disadvantaged individuals— table 91 of the 




ompioyi 1 loin sw-unry auiomuTea reponing sysrem 


State employment security agency 


Annual planning report 


State employment security agency 


State department of health (bureau of vital statistics)— 


State department of health 


c\c\\c\l 'rlntn nnolx/^ic 
uuiu/uuiu \j i rvjiy^'o 




Economically disadvantaged individuals— table 91 of the 


State employment security agency 


employment security automated reporting system 




Aid to families with dependent children 


County welfare offices 


School enrollment trends 


Local education agencies 


iionub iium social service agency aaminisiraiive aara 


Local social service agencies 


Ktryiui iui uiKj ivx^ui piunniny commissions— annual ana/ar 




special reports 


Regional and local planning commissions 


Annual planning report 


State employment security agency 


State department of planning and/or community 


State department of planning and/or community 


development— special studies 


development 


Current population report 


U.S. Department of Commerce 


State department of health (bureau of vital statistics)— 


State department of health 


data and/or studies 
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Defining the Context for Employment 

Information Element Question* to Be Answered Information Purpose 



Economic Status and Industrial Base What have been the area trends in To demonstrate whether a favorable 

unemployment^ economic climate exists for absorbing 

new vocational graduates and to 
consider effects on placement rates 



What are the significant local indus- 
tries and employers in terms cf loca- 
tion, employment, and growth 9 



To plot future status of employment in 
the community 

To identify sources of nonpublished 
information on employment trends 

To identify whether potential labor 
market problems, such as transporta- 
tion, exist 
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Defining the Context for Employment 



Information Sources 



Agencies Supplying Information 



Annual planning report 

Labor market information newsletter 

Affirmative action reports 

Characteristics of the insured unemployed— ES 203 report 

Demographic characteristics of those eligible for unem- 
ployment insurance and registered with the state 
employment agency-table 8 of the employment security 
automated reporting system 

Characteristics of the insured unemployed— ES 203 report 
Chamber of commerce— data or special studies 



Local- and state-level economic or industrial development 
commissions— data and/or studies 

Industrial realtors— information and/or data 

University departments of business and/or economic 
research; regional multistate councils of government, and 
employer associations— data and special studies 

Tax records, if local wage taxes are collected 



State employment security agency 
State employment security agency 
State employment security agency 
State employment security agency 
State employment security agency 

State employment security agency 
Local chambers of commerce 

Local- and state-level economic or industrial development 
commissions 

Local industrial realtors 

Colleges and universities in the state (or surrounding 
states), multistate councils of government, and local 
employer associations 

Local tax offices 



Annual planning report 

County business patterns 

Labor market information newsletter 

State department of industrial development— directory of 
major manufacturers 

Local chambers of commerce and regional and local 
planning commissions— data or special studies and 
employer directories 

State and area projections from the occupational 
employment statistics program 

Employment service 202 Report 



State employment security agency 

U S Department of Commerce 

State employment security agency 

State department of industrial development 



Local chambers of commerce and regional and local 
planning commissions 



State employment security agency 



State employment security agency 
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Assessing the Needs of Employers for Trained Workers 

information Element Questions to Be Answered Information Purpose 



Demand for Trained Workers what are the growth and decline To determine industries and occupa- 

pattems in the mix of occupations tions that are expanding and will 

and industries? have chronic or crucial needs for 

workers 



What are estimates of demand for To identify the potential size of the 

new workers, by occupation, requiring market for vocational education 
less than a bachelor's degree'? 
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Assessing the Needs of Employers for Trained Workers 

Information Sources Ag#nclei Supplying Information 



Annual planning report 
Employment service 202 reports 

Employment service 790 series— current employment statis- 
tics program 

Labor market information newsletter 
County business patterns 
Economic censuses 



State and area projections from the occupational 
employment statistics program 

Decennial census (for states and standard metropolitan 
statistical areas with a population of 250.000 or more) 

Openings received and filled by industrial division and 
occupational category— table 10-A of the employment 
security automated reporting system 

Special studies about the mix of an area's occupations 
and industries 



State employment security agency 
State employment security agency 
State employment security agency 

State employment security agency 

U.S. Department of Commerce 

U.S. Department of Commerce. Bureau of Census 

State employment security agency 

U S. Department of Commerce. Buieau of the Census 
State employment security agency 



State and local industrial or economic development 
commissions, chambers of commerce, regional planning 
commissions, and universities 



State plan for vocational education 
Annual planning report 

Table 96 of the employment security automated reporting 
system 

Job bank openings summary 
JOBFLO data 

State industry -occupation matrix from the occupational 
employment statistics program 

Decennial census publications 

Employer surveys 



Help-wanted ads 

Occupational projections and training data 



State division of vocational education 
State employment security agency 
State employment security agency 

State employment security agency 
State employment security agency 
State employment security agency 

U.S. Department of Commerce 

Chambers of commerce, planning commissions. JTPA 
sponsors, university research projects, employer 
associations 

Local newspapers 

U S Department of Labor. Bureau of Labor Statistics 
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Assessing the Needs of Employers for Trained Workers 



Information Etarwrrf 



Questions to Be Answered 



Information Purpose 



What has been the pattern of 
demand for vocational education by 
employers to tram current employees'? 



To assess the potential for developing 
special vocational programs for 
upgrading or retraining workers 



Supply of Trained Workers what ore estimates of tra.n.ng supply. To help ,dent,fy potential avenues for 

by occupation, to meet demands for providing vocational instruction 
new workers' 
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Assessing the Needs of Employers for Trained Workers 

Information Sources Agencies Supplying Information 



Contacts with local employers- determine current 
upgrading/retraining methods; training suppliers currently 
utilized; occupations for which training needs for current 
employees; satisfaction with hiring vocational education 
graduates 

Contacts with state occupational licensing bureaus and 
other training groups to determine areas of training 
shortfalls 

Contacts with employer associations and employee 
groups (unions) to determine upgrading and retraining 
needs 



Local private employers 



Table 96 of the employment security automated reporting 
system 

Vocational education data and reporting system 

Higher education general information survey 

National Center for Education Statistics* postsecondary 
career school survey 

Contacts with representative of state bureau of system 

Work incentive program completions— employment secur- 
ity automated reporting system 

Occupational projections and training data 

Job corps centers 

Contacts with and data from local proprietary tchools and 
community or junior colleges 

Contacts with local employers (especially for information 
about unregistered local apprenticeship programs) 

Employment service 203 report— unemployment by occu- 
pations for standard metropolitan statistical areas 



State bureaus of occupational licensing 



Local employer associations and employees groups 



State employment security agency 

State division of vocational education 
National center for education statistics 
National center for education statistics 

State bureau of vocational rehabilitation 
State employment security agency 

US Department of Labor. Bureau of Labor Statistics 

Local centers or U S. Department of Labor 

Local proprietary schools and community colleges- state 
agency responsible for community and junior colleges 

Local private employers 

State employment security agency 
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Assessing the Needs of Employers for Trained Workers 



Information Element 



Question* to fie Answered 



Information Purpose 



What are the sources of supply in this 
community and region? 



To account for effects of other voca- 
tional education institutions and 
related agencies in the community 
and the region 



oupply-Demand Balance What occupations have a large esti- To identify gaps that could conceiv- 

mated supply-demand gap, either by ably be filled by vocational 
numbers or by production? education 



What occupations have job openings To provide supplementary information 
for trained persons'? to assist in identification of shortfalls 
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Assessing the Needs of Employers for Trained Workers 



Information Sources 



Agencies Supplying Information 



Listing of proprietary schools (for locations of schools) 

Local school belonging to the National Association of 
Trade and Technical Schools 

Contacts with employer associations and employees 
groups (locations of in- house or employer-sponsored train- 
ing programs) 

Listing of colleges and universities 

Contacts with state bureau of apprenticeship and training 
representative; state bureau of vocational rehabilitation; 
and bureaus of occupational licensing (location o f train- 
ing programs) 

Local telephone directories 

Directories of local schools offering vocational education 



State agency responsible for proprietary school education 
and licensing 

National Association of Trade and Technical Schools. 
Washington. D C. 

Local employer associations and unions 



State department of higher education 

State bureau of apprenticeship and training, bureau of 
vocational rehabilitation 



State division of vocational education 



Annual planning report 

State plans for vocational education 

Table 96 of the employment security automated reporting 
system 

Locally developed estimates based on local supply- 
demand matrices, along with other selected criteria 

Occupational projections and training data 



State employment security agency 
State division of vocational education 
State employment security agency 



Local planning commissions, chambers of commerce, 
local education agencies, university research projects 

U S Department of Labor. Bureau of Labor Statistics 



Annual planning report 
Job bank openings summary 
JOBFLO data 

Local survey already conducted 



State employment security agency 

State employment security agency 

State employment security agency 

Local vocational education agencies, chambers of com- 
merce, civic improvement association, employer associa- 
tion, junior or community college, local private employ- 
ment agencies 



85 



9 

ERLC 



101 



Assessing the Needs of Employers for Trained Workers 

Information Element Questions fo Be Answered Information Purpose 



What are the apparent causes of any To provide supplementary information 
significant estimated or reported short- to assist in identification of shortfalls 
ages of trained workers? 
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Assessing the Needs of Employers for Trained Workers 

Information Sources Agencies Supplying information 



Contacts with local advisocy councils 



Contacts with industrial or economic development 
commissions 

Contacts with state occupational licensing bureaus and 
other employer groups 

Help-wanted ads 

Private employment agencies—special survey 



Labor turnover statistics programs 

Occupational wage rates based cn data from table 7B of 
the employment security automata reporting system, 
and job bank and JOBFLO data 

Bureau of labor statistics area wage surveys 

Employer requirements for worker education and expe- 
rienced based on JOBFLO data 

Employer job descriptions 

Seasonal employment trends based on data from the cur- 
rent employment statistics program of the bureau of labor 
statistics 790 program 

Selected employer interviews to determine working condi- 
tions, promotability within the fi.m and job requirements 

Occupational outlook handbook 

Dictionary of occupational titles 

Occupational projections and training data 

Contacts with state occupational licensing bureaus, labor 
unions, and employer associations 

Worker interviews with vocational education completers 
working in specific occupations to determine perceptions 
about job sotisfaction and working conditions 



Local education agencies and community colleges 

State and local economic/industrial development 
commissions 

Employer groups, state occupational licensing bureau 
Local newspapers 

Local private employment agencies 

State employment security agency 
State employment security agency 

U.S Department of Labor. Bureau of Labor Statistics 
State employment security agency 

Local private employers 

State employment security agency 

Local private employers 

U S Department of Labor 
U S Department of Labor 

U.S Department of Labor. Bureau of Labor Statistics 

State occupational licensing bureaus, local labor unions, 
and employer associations 

Completers of vocational education programs 
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Assessing the Needs of Employers for Trained Workers 



information Element 



Questions to Be Answered 



Hiring character! sties 



What occupations appear to offer the 
largest number or proportion of job 
openings to fresh graduates of voca- 
tional programs? 



Information Purpose 



To reduce the possibility of producing 
skilled people who won't be hired 



How are vocational education com- 
pleters perceived by employers in 
comparison with potential workers 
from other training sources 9 



To determine whether vocational 
education is seen as a viable source 
to help fill employment gaps 
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Assessing the Needs of Employers for Trained Workers 

Information Sources Agencies Supplying Information 



Contacts with local proprietary schools, community and 
junior colleges, and public vocational schools- 
placement and/or follow-up reports 

State divisions of vocational education 

JOBFLO data 

Help-wanted ads 



Local vocational school— employer follow-up surveys to 
determine satisfaction with vocational graduates 



Local proprietary schools, community and junior colleges, 
and public vocational schools 

State division of vocational education 
State employment security agency 
Local newspapers 

Local public vocational schools. State department respon- 
sible for vocational and technical educat'on 
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Assessing the Needs of People for Job Skills 

Information Element QustfJcns to Bs AnswsrGd Information Purpose 



Economic Needs What proportion of the population To determine the extent of the need 

can be described as economically for vocational education as an eco- 

disadvantaged? nomic improvement program 



What croups or subpopulations are 
suffering economic hardships 



To evaluate whether potential pro- 
grams need to be focused toward 
porticula regroups of clients 



Employment Needs What segments of the population are To describe the needs of individuals 

most affected by unemp'oymenP for skills that will enable them to 

obtain stable employment 



What proportion of the population is 
employed m jobs that appear to be 
sexually or racially stereotyped? 



To what extent do individuals training 
to be able to move out of occupa- 
tional "ghettos" 9 
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Assessing the Needs of People for Job Skills 

Information Sources Agencies Supplying Information 



Annual planning report 

State department of planning and/or community 
development— data and reports 

Decennial census publications 



Annual planning report 

Table 91 of the employment security automated reporting 
system 

Decennial census publications 

Administrative data from social service agencies (welfare, 
offices of aging, bureau of vocational rehabilitation, drug 
and alcohol abuse centers) 

County extension agents in rural areas 

Affirmative action report 

Regional, county, or local planning commissions— data, ar 
special reports 



Annual planning reports 
Affirmative action reports 

Table 91 of the employment security automate r feportmg 
system 

The ES 203 report for selected standard metropolitan statis- 
tical areas 

Special labor force estimates, studies on youth 
employment 



Decennial census publications 
Placement and follow-up reports 



Affirmative action reports 

Occupational outlook quarterly; publications of the 
women's bureau. U S. Department of Labor; handbook of 
labor statistics. U S Department of Labor, bureau of labor 
statistics 



State employment security agency 

State department of economic or community 
development 

U.S Department of Commerce. Bureau of the Census 

State employment security agency 
State employment security agency 

U.S Department of Commerce. Bureau of the Census 
Local social service agencies 

County extension agents 

State employment security agency 

Regional, county, and local planning commissions 



State employment security agency 
State employment security agency 
State employment security agency 

State employment security agency 

State employment security agency 



U S Department of Commerce. Bureau of Census 

State division of vocational education and local training 
institutions 

State employment security agency 
US Department of Labor 
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Assessing the Needs of People for Job Skills 

Information Element Questions to S# An tw#r#d Information Purpose 



Educational/Skill Needs What proportion of the labor force has To show the need for vocational edu- 

inadequate education levels? cation as a credential to help over- 

come employment barriers 



What are the needs for vocational 
education expressed by underem- 
ployed persons? 



To ascertain potential interest of per- 
sons in programs to improve their 
labor market potential 
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Assessing the Needs of People for Job Skills 

Information Sources Ao#ncl«s Supplying Information 



Decennial census publications U S, Department of Commerce, Bureau of the Census 

Public school dropout data State department of education 

Table 91 of the employment security automated reporting State employment security agency 
system 



Administrative client assessment data Bureau of vocational rehabilitation and other social ser- 

vice agencies 

Guidance counselor information Local public schools 
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Assessing System Capacity 
to Meet the Needs of Employers and Students 



Information Element Questions to Be Answered Information Purpose 



Institutional Description What kinds of institutions offer pro- To develop an inventory of local pro- 

grams of instruction in vocational viders of vocational education 

education' 7 



What types of facilities are available 
for the vocational program'* 



To assess the capacity of local edu- 
cation agencies in terms of space 
and enrollment potential 
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Assessing System Capacity 
to Meet the Needs of Employers and Students 



Information Sources 


Agencies Supplying Information 


State directory of local education agencies offering voca- 


State division of vocational education 


tional education 




State directory of proprietary schools 


State division of vocational education 


National Association of Trade and Technical Schools— 


National Association of Trade and Technical Schools. 


school catalogue 


Washington, D.C. 


Encyclopedia of associations 


Local libraries 


State bureaus of occupational licensing (locations of 


State bureaus of occupational licensing 


accredited training programs) 




Yearbook of higher education 


Local libraries 


Programs and schools: a supplement to the directory of 


Local libraries 


post secondary schools with occupational programs 




Technician education yearbook 


Local libraries 


State directories of universities, community colleges, and 


State department responsible for higher and junior college 


technical institutes 


education 


State representative of bureau of apprenticeship and train- 


State bureau of apprenticeship and training 


ing (locations of local, registered programs) 




Local hospitals for specialized modical training 


Local hospitals 


Employer associations for employer in-house training 


Local employer associations 


programs 




Reports on enrollment capacity, spacing and equipment 
from local public schools, junior or community colleges, 
and other training institutions 

Facilities and equipment information 
Higher education data survey 


Local schools, junior or community colleges, and other 
local training institutions 

State department of education, state division of voca- 
tional education 

State department of higher education or National Center 
for Education Statistics 
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Assessing System Capacity 
to Meet the Needs of Employers and Students 



information Element 



Questions to Be Answered 



information Purpose 



Program Description 



What is the range of occupational 
programs offered at each institution? 



To describe the total range of voca- 
tional options available in the area 



What are the employment objectives 
of instructional programs at each 
institution? 



To determine which institutions are 
involved in specific job preparatory 
instruction rather than career orienta- 
tion, exploration, retraining/upgrad- 
ing. and so forth 



Student Body 



At each institution, what are the levels 
of enrollment (and trends) in voca- 
tional programs? 



To indicate the trends in student 
interest toward various types of 
programs 



What are the patterns of completion 
of vocational programs, by occupa- 
tional area? 



To provide estimates of occupational 
supply 
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Assessing System Capacity 
to Meet the Needs of Employers and Students 



information Sourcts 



Agencies Supplying Information 



Catalogues from local community colleges or technical 
institutes 

Contacts with directors of occupational planning or institu- 
tional planning at community colleges or technical 
institutes 

Catalogues or brochures from proprietary schools or con- 
tacts with administrative staff 

Vocational education data and reporting system 

Directors of local Job Corps centers 

Training directors— local hospitals and military installations 

State bureau of apprenticeship and training— listings of 
local programs 

Contact with directors of sheltered workshop programs 



Contacts with a knowledgeable person in each training 
institution (e.g., training directors, institutional planners, 
instructors, school administrators, recruitment specialists) 



State department responsible for community colleges and 
technical institutes 

Local community colleges and technical institutes 



Local proprietary schools 

State division of vocational education 
Job Corps centers 

Local hospitals and military installations 
State bureau of apprenticeship and training 

Local sheltered workshops 



Local training institutions 



vocational education data and reporting system 
Enrollment records 



National Center for Education Statistics— postsecondary 
career school survey 

Enrollment records— local community and junior colleges, 
local proprietary schools, apprenticeship programs, local 
sheltered workshops, Job Corps centers, and any other 
local training groups 



Vocational education data and reporting system-follow- 
up component 

Completion and/or follow-up reports local community or 
junior colleges and proprietary schools 



Job Corps centers completion records 



State division of vocational education 



Local community and junior colleges direcrly responsible 
or state department responsible for higher education 

National Center for Education Statistics 



Local community and junior colleges, proprietary schools, 
state bureau of apprenticeship and training, local shel- 
tered workshops, local Job Corps centers (or state 
department of labor), and other training institutions 



State division of vocational education 



Local community and junior colleges (or state department 
responsible for higher education), local sheltered 
workshops 

U.S Department of Labor, employment and training 
administration or local centers 
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Assessing System Capacity 
to Meet the Needs of Employers and Students 



Information Element Questions to Be Answered Information Purpose 



Who? are the age. socioeconomic. To determine what types of persons 

and employment characteristics of are in fact being served by local 
students enrolled in training? institutions 



What are typical entrance require- To determine how accessible pro- 

ments to vocational education gram* are to persons of differing abil- 

programs? ily levels 



Financial Status What have been the trends in funds To show potential for expansion of 

available for operational expense of program offerings 
vocational education? 
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Assessing System Capacity 
to Meet the Needs of Employers and Students 



Information Sources 


Agencies Supplying Information 


\Avntionnl AHiif*ntir>n rtntn nrvH rorwtinn cuctom 


State division of vocational education 


Individual school nrvi plnss mrvvt^ 






Local schools and training instructors 


Individual community and junior college institutional 


Local colleges; state department of education and/or 


records and reports 


higher education 


Higher education general information survey 


National center for education statistics 




Local proprietary schools 


Job Corns centers— stuctont hnriv rftrwts 


Local centers 


vJIUUt7lM Uw"— I0|JUI 13 IIWITl 9llt7(lt7lt7U WUf K5M lOD!> 


Local sheltered workshops 


Vocational rehabilitation management information system 


State bureau of vocational rehabilitation 


State plans for vocational education 


State division of vocational education 


Contacts with local vocational school administrators, 


Local vocational education agencies 


vocational guidance counselors, or individual instructors 


Catalogues from junior and community colleges 


Local community and junior colleges 


Catalogues or brochures from local proprietary schools 


Local proprietary schools 


Contacts with state bureaus of occupational licensing 




Contacts with admissions specialists, recruiters, or counsel- 


State bureaus of occupational licensing 


ors at community colleges, proprietary schools, and other 




training institutions 


Local training institutions 


State plan for vocational education 


State division of vocational education 


Local school budgets and financial reports 


Local schools 


Higher education general information survey 


State department responsible for higher education, state 




occupational information coordinating committee 




(SOlCC) director. National Center for Education Statistics 


State budget legislative reports 


State agency responsible for fiscal matters 
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Assessing System Capacity 
to Meet the Needs of Employers and Students 



information Element Quettlon. to 6* Answered Information Purpor 

What Is the availability of funds for To show potential for capital and 

capital expenses (facilities, 'equip- equipment expenditures 
ment) in vocational education? 
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Assessing System Capacity 
to Meet the Needs of Employers and Students 



Information Sourest 



Agencies Supplying Information 



Local school equipment inventory expense records 
State plans for vocational education 



State education agency budget allocations for facilities 
and equipment 



Local school district budget records and reports 



Local schools 



State division of vocational education 

State department of education, state division of voca- 
tional education, state department responsible for higher 
educat'on 

School district administrative offices 
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